Sunday, December 14, 2008

Users and cron jobs

Today, we implemented some basics: User creation and ticks. The results can be found here.

After login (using Google's user authentication), we require that a nick name is chosen. From Google we only get the email-adress of the current user as an authentication token - obviously not perfect to address the user.
While making sure that an already used nick cannot be chosen, we found out that RequestHandler.redirect(url) does not really redirect the current call. Rather, it sets the "Location" header of the HTTP response and then returns.

Additionally, we implemented the ubiquitous "tick". Every five minutes resources are recalculated, ships are moved and so on. Since we cannot use the cron job from GAEUtilities, as noted in a previous post, we implemented our own lazy cron scheduler. During each request we first update the current tick from the start date of the round and the current time.
Then, for each cron job we determine, if one or more ticks have passed since the last time we executed it. If so, we run the job and provide the number of times it ought to have been called by now. Having processed all cron jobs, the actual user request is processed.

Using this implementation, we now have available a primitive resource distributer. For each tick, credits and metal are increased by one.

Next time, the Control Center will be awakened. :)

4 comments:

Joe Bowman said...

Hi guys. Interesting post on what you're doing. I'm the guy behind gaeutilities and the cron job. I thought I'd drop you a note and let you know a consideration I took account for when building Cron, so you can be sure you do the same for your application.

The request timeout for GAE is pretty short. What I did for Cron was put in a time. I get the time when I start processing cron job entries, and at the end of each job, I check to see how much time has elapsed since I started running jobs. If it is >= 1 second, I go ahead and stop running jobs, leaving the rest of the timeout period for the processing of the actual user request.

This way the user shouldn't be presented with any application timeout errors imposed by GAE, provided of course your actual request isn't heavy enough it needs the full process time.

Interesting posts on your GAE development strategies, good luck with your project.

T-002 said...

Hi Joe,

thanks for your input. I've thought about this some, but I don't think its viable in our (special) case.

Your intention here is, as you mentioned, to get the user request answered within the GAE timeout and obviously we'd love to do that, too. Unfortunately, the tasks to be performed are not of the kind that can be postponed until later. They ensure the programs progress and more importently: the datas consistency.

Consider the following example: Between requests 13 minutes pass by. In this time, we should have provided each user with an additional 500 credits, as the tick in this case is 5 minutes. If we manage to perform only the first update of user's accounts we'd actually deprive the currently requesting user of considerable resources and therefore simply of options. And we didn't yet talk about confused users ...

In this situation the only option we have, I think, is to "suspend" the request before the GAE timeout and return a static, hopefully helpful error page.

In the end, I'd like to return the favor and propose a feature to Cron: Make the timeout configurable, allowing it to be turned off. :)

All the best,
T-002

Joe Bowman said...

I've run into similar issues on the project I'm working on. In fact I'm not even using my own cron utility, and have moved to issuing requests from a separate server. Basically doing a process where I pull data from my site, process it, and return what's necessary. Offloading a lot of CPU overhead.

I've added an issue to the project to take a look at your suggestion for adjusting the timeout for Cron.

murat said...

@Joe

is this second server outside GAE? It probably is, otherwise you couldn't reliably call your site. The problem I see is that you loose the major advantage of GAE: leveraging Google's infrastructure and its distributed (and free) computing power ...

regards,
murat