Solved

Waiting for multiple parallel jobs on workflow

  • 25 April 2023
  • 1 reply
  • 309 views

Userlevel 1
Badge +3

Hello, i created workflow and use foreach for run backup task (multiple clients). how i can wait job complete for multiple parallel jobs?

i tried use WaitForJobCompletion in step after starting backup task. If configure "number OfParallelExecutions" value in "1" for foreach, task is executed sequentially (it's normal,  we are waiting for the task to complete to start another task).

When configure "number OfParallelExecutions" value in (for example) "10"  for foreach and use WaitForJobCompletion in step after starting backup task, the tasks are executed in parallel, but we have error "Error Code: [19:1805]" in workflow. But, workflow work correctly and backup done.

finnaly, is there a way to monitor (for jobs waiting for other steps on the workflow) the execution of multiple parallel  (backup) jobs runs inside a "foreach" block outside of "foraech" block? When uses "number OfParallelExecutions" value in (for example) "10"  for "foreach" block.

Thanks

icon

Best answer by RMcG 9 May 2023, 06:38

View original

1 reply

Userlevel 3
Badge +6

Hi,

If you want to run a maximum number of jobs in parallel then you should add the WaitForJobCompletion activity inside the ForEach loop. In this way the numberOfParallel executions will control how many jobs can be running at once. As each iteration completes the thread will free up allowing another to start until they are all complete.

If you are getting an error with this approach then at a guess it’ll be to do with collecting the job id and passing it to the WaitForJobCompletion activity … should be able to tell by checking workflowEngine.log

The jobId input for the WaitForJobCompletion task accepts a list of job ids. This means you can also create multiple jobs and pass a list of the job ids to the activity. For example you could use the foreach loop to submit the jobs and then monitor those jobs for completion after they have all been submitted. In this case the numberOfParallelExecutions will control the maximum number of new job requests that can submitted in parallel but the loop won’t wait for any to complete eg it’ll just submit each and move on to the next.

Example of this:

Inside the foreach loop…

  • start the job
  • get the job id returned
  • add the job id to a list outside the foreach loop (you can create an List<Integer> workflow variable for this)

You then add the WaitForJobCompletion after the foreach loop. Set the jobId input to the workflow variable you are storing the job ids in.

 

Hope that helps

Reply