Admiration/Advice
|
docmattman
Joined: 2003-09-11
Posts: 12 |
Posted: Thu, 2006-06-29 05:39
|
|
Gallery is great. There is a specific part of it that I am particularly interested in and I was wondering if I could get some "how to" advice... During the import process to bring G1 into G2, I noticed that you seem to have some sort of process that bypasses the PHP default max_execution_time setting to accomidate for a large import. During this process you also have a nice progress bar system (looks like javascript) that tells the current process/status of the import. I was wondering if someone could give me the basic rundown of the steps taken to accomplish this task as I am interested in doing something similar to this (with different context). I want to do some PHP processing, that could take a long time to complete, but I also want to display the progress/status at the same time. Can anyone offer suggestions for creating this working process? |
|
Posts: 32509
there are several approaches to run long tasks on a webserver via a PHP frontend.
the one used in G2 works like this:
- the task usually consists of a loop doing XXX iterations of doing something.
- in every loop iteration*, we extend the script timeout and the webserver timeout for another few seconds.
- in every loop iteration*, we send an update of the progress to the browser (progress bar)
- in every loop iteration, we do some work of the actual task.
- in every loop iteration*, commit your work.
*) if a single loop iteration takes too few time, you can use a modulo counter to only do these things every 10 / 100 / ... iterations.
notes:
1. a webserver is an unreliable environment. using transactions is a very good idea.
also, periodically using checkpoints (committing the current transaction and starting a new one for the rest) for very long tasks may be a good idea for certain applications.
2. when running PHP on top of apache or IIS, you basically have to fight 2 timeouts. php's max execution time and the webserver timeout.
to fight php's timeout, you can dynamically set the time limit at runtime with set_time_limit(...) (see http://php.net/set_time_limit).
to fight the time limit of the webserver, you have to keep sending some data to the browser the whole time. usually the webserver stops the process after 300 seconds (apache default, can be changed).
3. and you want to inform the user of the progress so far.
2. and 3. can be addresses with the same feature, the progress bar. our implementation of the progress bar is very simple and based on HTML / JavaScript.
what we basically do is this:
- user starts a long running task, e.g. the g1 -> g2 import
- we immediately return the first part of a HTML page to the user's browser. the HTML consists of a <table> of 1 row with 2 columns. the 1 column has 0% width and orange color, the second has 100% width and grey color.
the HTML also contains a javascript function which updates this table. the javascript function changes the widths of the 2 columns of the table when we call it.
- we start running the g1 -> g2 import task. after 2% of the work, we send some HTML to the browser. it's the same connection, we just never stopped sending data to the browser.
the HTML that we have sent is just a javascript call. it calls the function that we defined earlier with 2 such that the javascript updates the table columns to be such that the first column has a width of 2% and the 2nd column 98%.
- after 5% we send again a javascript update to the browser, etc. etc.
you can just look at the HTML source code of the g1 -> g2 import page once it is finished loading.