#80 ✓hold
joep

enable modules to create worklings...

Reported by joep | August 17th, 2010 @ 07:33 AM

For the first module I'm doing I needed to be able to create worklings. The attached two line patch allows for modules to have worklings. It would be nice if they were started only when the module is actively used, but no simple solution appeared to me based on what I know of the code at the moment.

Comments and changes to this ticket

  • Pascal Rettig

    Pascal Rettig August 17th, 2010 @ 07:50 AM

    • State changed from “new” to “open”

    What do you need the worklings to do? I'd rather not tie the modules into a specific background processing solution as that may change down the line,
    DomainModel has a way to run arbitrary functions using the DomainModelWorker, which ensures that the correct Domain DB is activated, you can run it manually:

    To run a class level method, use:

    DomainModel.run_worker("WorkerClassName",nil,"worker_method",args) which will run:

    WorkerClassName.worker_method(args ={}) in a background process, of you have an instance of a DomainModel you can run,

    MyModel.run_worker('method_name',args) and that will run an instance method on the object after finding it by id

  • joep

    joep August 17th, 2010 @ 08:39 AM

    Well, that almost works. The problem is that these worklings only have one thread. (It would be nice if they could manage a thread pool for themselves.)

    What I am building (and it is just a first attempt at the moment :)) is a module to handle complex graph rendering paragraphs. The paragraphs can take some arbitrary time to generate, up to a couple of seconds, during which time the DomainModelWorker would be tied up, if I read everything correctly.

    Alternative suggestions always appreciated, of course.

  • joep

    joep August 17th, 2010 @ 08:51 AM

    I forgot to mention that the next thing I'd like to do is generate PDF files in the background that includes some other graphs based on a user request (and then sell it in the store, but that is another topic :) ), which again could take a little bit of time.

  • Pascal Rettig

    Pascal Rettig August 17th, 2010 @ 09:47 AM

    Hi Joe,

    One of the reasons why I don't want to tie modules directly into the worker system is because we are planning on rewriting it to use multiple threads per worker class (up to some limit) for exactly that reason.

    There are two options as a workaround for now:

    1. We create a 'SlowRunningWorker' that does the same thing as the main DomainModel worker but should only be used for non time dependent things - this will keep the DomainModel worker running smoothly for processes that only take a couple of seconds

    2. Enable the multiple option in script/workling_client and you can start multiple workling on the same machine to distribute the load,
      just run: ./script/workling_client start repeatedly and it'll start more than one up. ./script/workling_client restart doesn't seem to work correctly for this however, so you'll need to run a stop and multiple starts separately.

    -Pascal

  • joep

    joep August 17th, 2010 @ 11:01 AM

    For now I'll go with option 2, since it is the least work for both of us now and in the future, even if it is a little wasteful.

    Depending on timing, at some point it may become necessary to continue down this path, but I'll weigh that against forward compatibility when the time comes.

    It seems appropriate to close out the ticket though.

  • Pascal Rettig

    Pascal Rettig August 17th, 2010 @ 01:17 PM

    • State changed from “open” to “hold”

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

Shared Ticket Bins

People watching this ticket

Attachments

Pages