sales report takes too long, stathandler.cgi running on

[MServitus]

Every thing was working fine until yesterday.
Then suddenly when we tried to view our orders for the day,
the browser indicator just spins and spins. It finally
comes back with a proper screen after 3 minutes.
During that time our server shows that the program stathandler.cgi is eating up 50% of its CPU. And nothing else can get done.

I did some reading on this forum and found a hint that this might be caused by a corrupted orders.stats.db file.

Could that be my problem.? How can I correct this problem.?? Delete the orders.stats.db file.??

Is there another possible cause.??

MS.

[loren_d_c]

Go to the Configure Reports menu to see the path and filename of your webserver's log file that it is configured to read, and check the filesize of that file. If it is over 50 Meg or so, consider rotating it. And it if gets this large often, consider setting it up to be rotated on a weekly basis via a cron job. Your host should be able to help you with that if need be.

-Loren

[MServitus]

Thanks Loran for the input.
the orders.stats.db file is 11,881,984 blocks. far from 50 meg.
also, what higher menu is the "Report configuration" menu under.??

Give the 11 meg size, is there some other possible explanation?

also, you suggested "rotating" the db file. Do you mean to rename the existing and allow Shopsite to re-create it.??

Or some other method.??

MS.

[loren_d_c]

The orders.stats.db is not the file I was asking the file size on, nor was I suggesting rotating this file.

For the traffic portion of the Reports, ShopSite parses your WEBSERVER's hits log file. This file is not created by ShopSite, only read by ShopSite. It is created by your webserver, which adds an entry to it each time there is a hit on your website. The location of your site's access log file can vary depending on the server setup, so this is why I suggested going to the Configured Reports menu to see the location of the file that your ShopSite is configured to read so that you can locate it and check the size.

The Configure Reports menu can be reached by clicking on the Configure Reports button on the main Reports menu.

-Loren

[MServitus]

and it is big 2.66 meg. I didn't realize Shopsites report stats are taken live from the log file.
Later today during non-prime time, I will knock off the back 3/4s of the file and try your suggestion.
I have been keeping this file around and letting it get too big 'cause I have been using it to measure the traffic loss we took during migration to a new machine.

MS

[loren_d_c]

Do you mean 2.66 Gig? 2.66 Meg wouldn't be very big at all.

Only the Traffic portion of the Reports is parsed from the webserver traffic log. If you have some other web traffic analysis tool that you use for your traffic reporting (Google Analytics, Urchin, Analog, Webstats, awstats, etc) and you don't need/care about the traffic portion of the Reports, then you could point ShopSite's Reports feature to a 0-byte dummy file and it would take no time at all to just show you the Sales side of the Reports.

-Loren

[MServitus]

Ok I shrunk the access_log file down from
2,663,614,288 to 3,395,553

Then clicked to go to the reports screen. The
browser started spinning and stayed there for 10
minutes. Then the process terminated with this error message:

Server encountered an internal error and was unable to complete your request.

Error Message: Premature end of script headers: Stathandler.cgi

Now I can't get a report to come up at all..!!

So, based on Loren's last posting, I zeroed the file out and tried it again. Now at
8 minutes and still spinning. Looks like my problem is not related to the size of the access_log file.


MS

[loren_d_c]

After you shrunk and/or zero'ed out this file did you also restart the apache webserver? The webserver has this file open always so sometimes things you think you are doing to it don't take effect unless the webserver is briefly stopped.

-Loren

[MServitus]

yeap. I restarted apache. That got data flowing into the log file again.
But Report still spins the browser indicater for approx 10 minutes then times out.

MS..

[loren_d_c]

You could try the Empty Store Log and Empty Order Log functions on the Reports Configuration screen (if you can't get to this menu through the Reports menu, you can go directly to it by going to [your_ShopSite_backoffice_url]/statlogconfig.cgi).

The Empty Order Log should overwrite orders.stats.db with an empty one, and Empty Store Log gets rid of the file in the data/stats/ directory where relevant hits from the webserver log are copied and kept for 30 days. These are only used by the Summary features (Page Summary, Host Summary, Referrer Summary) of ShopSite Pro, and they should not affect the main traffic and sales counters (just don't accidentally hit the Clear Sales Counters or Clear Traffic Counters buttons).

-Loren

[MServitus]

Just to be save, I downloaded orders before I did it. Clearing the counters and emptying logs worked . Report screen now works quickly. I placed an order and insured it came through properly and was added to the just-zeroed counters.

I'm a happy customer again..
Now for the Next issue...

MS