Some of the trends we are seeing in our user base include the leveraging of EC2, larger apps/user bases, demanding HA requirements and hardware farms. As a result, scaled-out architectures are becoming common place in many IT environments. Scheduling work in these large multi-node environments is becoming a part of many software developers lives. Currently, Quartz supports scale-out but with little control over how the work is distributed. While this is a good start one quickly runs into problems like assigning jobs to machines that have the processing power to the work on them at the time the job is fired. Or execute the job where the data is local. Or just perform certain jobs on certain classes of machines do to their location or by purpose.
Well that is "where" we are headed. In the next few months the Quartz guys working with the Terracotta Guys and the Ehcache guys are developing a solution to the above set of problems. We'll be giving the same flexible and reliable scaled out scheduler but adding a new level of control. We are adding the "Where"
Stay tuned...
Posts
2 comments:
Good move, locality of processing is very important both as a way to properly use CPU power and to achieve higher scalability by moving processing toward data.
Keep up the good work,
Cheers,
Sergio B.
Steve, trying to build a multi-tenant scheduling system for a SAAS environment. What's your take on best approach? We haven't considered Terracotta, while Quartz is almost guaranteed to be used. We have a currently implemented system which needs to be migrated to a new clustered environment, to give you the full picture
Post a Comment