Hello,

I struggle with a problem 2 weeks now and I�m trying to find the best suitable solution.
The problem:
Real time reports generated from logs from different devices. Some of the devices are Windows, Linux, switches etc.
The problem is the large volume of data; we are talking about several millions of data per day, per device type. So a monthly/weekly report generated (including almost live data), will be heavy especially if you are inserting data on those tables at the same time. The reports will be shown in an internal website and the requirement is them to be generated in near 10 seconds.

After reading some articles and searching the web I found several ideas of how to deal with the problem, but I�m not sure what�s the best path. I would like to hear some new suggestions or ideas or give me a feedback on the solutions found below.

Now I�ve read about splitting past data to different tables (partitioning), tables including the report�s data ready for read (without group by) updated constantly by triggers on live tables, real time data warehouse, programs having report�s data in memory for fast access and processing.

Also I want to mention that I�m using MSSQL server 2008 clustered on a powerful storage.

Thanks for your time,
Harris