   [web] Collecting and displaying statistics
  I'm a game developer without much experience of web development. I have something I'd like to set up which I think must be a fairly common sort of thing for web developers but I don't really know what the best way to go about it would be. I'd like some advice on the best way to set this up.What I'd like to do is set up an automated system that collects various statistics (performance statistics, memory usage statistics, etc.) from an automated nightly build of a game and posts them to a central server which can display the data in the form of various graphs, charts and tables in a browser. For example, the system could display a graph of how the average, minimum and maximum frame rate for a particular timedemo has varied over the course of several builds, or display a chart of how memory usage has varied in a particular level over time.I don't know much about databases but I'm guessing it might be a good idea to post the data to a database and query the database to generate the web based reports and graphs. Is this something that's fairly easy to do? What software could be used to generate the graphs etc. in a web-viewable format?Here's some things I'd like to be able to do with the system:- Being able to easily get the data into Excel for more detailed analysis would be useful.- Over time as the game is developed it should be easy to add new statistics and reports as new code is added to report on new features.- The system should be fairly easy to set up and maintain, it's probably not worthwhile if it takes a significant amount of work to implement.- We already have licenses for most MS software (Visual Studio, Office, SQL server) so a solution that doesn't require additional software would be nice (unless the software is free / open source)Any suggestions on good ways to implement something like this gratefully received.   
  I use perl for this sort of thing with our project.  Generally I write out the log data toa file, and use perl to parse it and put it into excel.  Other perl scripts can then be used to script Excel into generating charts and saving them to the htdocs folder of apache.From there you can have perl modify place holder tags in an html document that then becomes a live html document in the htdocs folder as well to refer to the graphs generated as well as any stats you wish to parse.Here's a sample charting section of code:$rangeID = "b2:b3";$Range = $writeSheet->Range($rangeID);$Chart = $Excel->Charts->Add;$Chart->{Type} = xlBar;$Chart->SetSourceData({Source => $Range, PlotBy => xlRows});$Chart->SeriesCollection(1)->{Name} = "first data set";$Chart->SeriesCollection(2)->{Name} = "second data set";$Chart->{HasTitle} = 1;$Chart->ChartTitle->{Text} = "Name of Chart";$Chart->Axes(xlValue)->{HasTitle} = 1;$Chart->Axes(xlValue)->AxisTitle->Characters->{Text} = "axis label";$Chart->Axes(xlCategory)->{HasTitle} = 1;$Chart->Axes(xlCategory)->AxisTitle->Characters->{Text} = "axis label";$filegraphname = "C:/web/Apache2/htdocs/NBUser/flash.gif";$Chart->Export({FileName   => $filegraphname,FilterName => 'GIF',Interactive  => 0});Sample for writing to Exceluse Win32::OLE qw(in with);use Win32::OLE::Const 'Microsoft Excel';use Data::Dumper;$Win32::OLE::Warn = 3;my $data = "c:\\scripts\\nbstats\\work";die "Need to create found.txt first.  Try running returnFiles.pl!" unless opendir( DATA, $data );my @Files = readdir(DATA);closedir( DATA );my $excelLog = "c:/Scripts/nbstats/excel/nbNewStats.xls";my $Excel = Win32::OLE->GetActiveObject('Excel.Application') || Win32::OLE->new('Excel.Application', 'Quit'); my $Book = $Excel->Workbooks->Open($excelLog); my $worksheet = $Book->Worksheets("OS Stats");my $row = 0;foreach my $Files (@Files) {    if( $row > 65000 ) { next; } # excel only allows around 65000 rows, atleast until the newest version comes out.my $fileName = $data . "\\" . $Files;if( $Files eq "." or $Files eq ".." or -d $fileName ) {next;}die "Can't open $fileName\n" unless open( IN, $fileName );my @text = <IN>;close( IN );foreach my $line (@text) {# code to do stuff$worksheet->Cells($row,1)->{Value} = $hres;$worksheet->Cells($row,2)->{Value} = $vres;$worksheet->Cells($row,3)->{Value} = $os;$worksheet->Cells($row,4)->{Value} = $osName;$worksheet->Cells($row,5)->{Value} = $nb;$worksheet->Cells($row,6)->{Value} = $pp;$worksheet->Cells($row,7)->{Value} = $flash;$worksheet->Cells($row,8)->{Value} = $ba;$row++;}$Book->Save;$Book->Close;CheersChris   ;  I've never used Perl but it looks like your method should be relatively easy to do in something I'm a bit more familiar with, like Python or C#. I quite like the idea of using a script to push the data into Excel and then scripting Excel to generate the output - that way it's easy to take advantage of all Excel's features to analyze and present the data. I think I might play around with something like this - from your example it looks fairly straightforward which is nice.Thanks for the info.   ;  The easiest way to push data to excel is a HTML table. Just generate a simple HTML file with only a HTML table in it, save it with the extension .xls and output it with a proper mimetype header. Excel will read the files just fine.   