Final GSoC report

What do I have completed this week?

  • Added a dedicated threadPool for the Processing Toolbox to run the algorithms.

  • Bugfix on the cancel option. Avoid connecting  the signals every time the algorithm runs. This was making the slots to be called multiple times when running and cancelling the algorithm several times.

Is there any blocking issue?

  • I wasn’t able to fix the issue with the cancel option when you have more than one algorithm running. The issue turned out to be more complicated than I first expected.

Advertisements

Parallel execution of QGIS algorithms

This is just an example of the advantage provided by the multithreading on the Toolbox. As each algorithm is running in a separate thread, this allows us to run more than one algorithm at the same time.

 

I’m polishing the implementation and trying to find some major problems in the current implementation.  It has some minor bugs that can be easily solved by adding a parameter to the  cancel signal (e.g. when you have two algorithms running and try to cancel one of them, the signal emitted will cancel two algorithms instead of just one)

Twelfth GSoC report – Multithreading on Processing

What do I have completed this week?

  • I have finished the cancel option in order to stop the execution of non-QGIS algorithms. It’s important to remember that as well as the QGIS algorithms the cancel option do not cover all the non-QGIS algorithms and this implementation only covers GDAL and SAGA algorithms. However this option can be easily replicated to the othersthird party algorithms.

  • I have changed the python console to use the QThreadPool.

  • I have been testing and debugging the multithreading implementation looking for unexpected behaviours or incorrect results.

  • Core refactoring.

What am I going to achieve for the next week?

  • Write documentation for the multithreading implementation.

  • Code cleaning and refactoring.

  • Search for problems in the multithreading implementation and correct them.

Is there any blocking issue?

  • No.

Eleventh GSoC report – Multithreading on Processing

What do I have completed this week?

  • During this week I have changed the multithreading implementation to use the QThreadPool to allow thread recycling and avoid thread creation costs every time we want to run an algorithm. This new implementation adds some flexibility to the multithread support and avoid unexpected behaviours in different machines. The number of threads in the QThreadPool is initialised accordingly to the number of cores in the PC and may happen that the QT is not able to detect the number of cores. In this case we will only have one thread in the thread pool, which can be occupied with something else. I’m trying to detect when there is no available threads on the ThreadPool and increase the number of threads if necessary.

What am I going to achieve for the next week?

  • Start working on a test suite for the multithreading to ensure the correct behaviour of the multithreading implementation. The test suite must check if the worker thread is starting, check if signals are triggering the functions when it is supposed to and verify if the cancel option is working properly.

  • Figure out how to stop the execution of a non-QGIS algorithm.

Is there any blocking issue?

  • I have spotted some issues when testing the code in a different machine (with Linux Mint 17) that may have to do with thread unavailability.  Therefore, I decided to postpone the current tasks and focus on changing the multithreading implementation in order to correct this kind of unexpected behaviours.

Tenth GSoC report – Multithreading on Processing

What do I have completed this week?

  • During this week I worked on a mechanism to stop the algorithm execution on QGIS algorithms. The approach used to cancel the execution explores python exceptions and QT signals to stop the algorithm from the main thread without having to wait in order to stop the execution and terminate the worker thread. I have used signals to trigger an exception inside the algorithm and stop the execution from the main thread. Since there is a significant amount of algorithms, the mechanism had to be implemented in just two of the QGIS algorithms. This implementation can be further replicated  towards the remaining  QGIS algorithms.

What am I going to achieve for the next week?

  • Start working on a test suite for the multithreading to ensure the correct behaviour of the multithreading implementation. The test suite must check if the worker thread is starting, check if signals are triggering the functions when it is supposed to and verify if the cancel option is working properly.

  • Figure out how to stop the execution of a non-QGIS algorithm.

Is there any blocking issue?

  • No blocking issues

Ninth GSoC Report – Multithreading on Processing

What do I have completed this week?

  • Solved an issue that makes QGIS crash when the main thread doesn’t wait for the worker thread to start.
  • I spent most of the time looking up some tutorials about unit testing on QGIS and trying to configure python paths to run the processing tests.

What am I going to achieve for the next week?

  • This week I’ll change back to the multithreading implementation in order to add the cancel option to stop the algorithm execution. I will be working on a mechanism to stop the execution on QGIS algorithm.

Is there any blocking issue?

  • This week my mentor proposed to start working on a suite test and I changed the plan to start working on a unit test to ensure the correct behaviour of the multithreading implementation. At the end of the week we discussed about asking the community to test the new implementation and report the issues if there is any.

Eighth GSoC Report – Multithreading on Processing

What do I have completed this week?

  • Created a mechanism to hold the result of the algorithm taking into account that there can be several algorithms running at the same time. The results are stored in a global variable called algResults. This variable is accessible via python console (e.g. “processing.algResults”).

algResults

  • The signals are connected in a different way in order to close the dialog and quit the thread when the algorithm is finished.
  • I have updated my fork and merged my work into the master branch.
  • Added the runalgIterating method to the AlgorithmExecutor with some changes to support the signals to set the progress
  • AlgorithmDialog refactored.
  • I have been searching for a solution to stop the algorithm execution and I think that is possible to create a sort of checkpoints inside the algorithm to check if we want to stop the execution and raise an exception to stop the thread event loop.

What am I going to achieve for the next week?

  • Allow the user to cancel the execution when running the runalgIterating.
  • Solve the problem with QGIS dependencies on Ubuntu and test the refactored algorithms.
  • Continue working on the algorithm iterating.

Is there any blocking issue?

  • I had to postpone the tests due to some problems installing QGIS dependencies on Ubuntu.
  • Didn’t had time to implement a solution to stop the execution of the algorithm.