database design question - a lot of fields or a lot of tables?

[GolferGuy]

I would combine all Value tables into one by adding a Company as a column. Same with all Growth data into one table with Company as a column.

As soon as I saw table names like IBM, GE, etc. the first thing I thought of was "3rd Normal Form"? I don't think so. Yes, Company NEEDS to be a field name in one table. I'm also wondering how much duplication of data there will be between the growth table and the Value table. I'm going to assume a lot of duplicate data.

I would highly recommend you find someone that truly knows how to design a relational database and pay them a little bit of money to help you design this system. It will be worth every penny you pay them if they get it right. Therefore, make sure they know database design before you hire them.

[mountainclimber]

GolferGuy - what would be duplicated in the Value and Growth tables? I can't think of a single thing that would be duplicated. Wait, now that I am looking at my last post I see where you may have been lead astray. This is what I should put as my sample:

tblIBM-Value <-company/entity specific "IBM" and group specific "Value" group table
One row per date (daily)
One column per entity attributes:
Price/Book <- a value for IBM
Price/Earnings <- a value for IBM
GICS Sector < - ID to a GICS Sector name table (which is not counted in my 33,000 table count)
...etc. to the N'th data item/column for the Value group

tblGE-Value <-company specific "GE" and group specific "Value" group table
One row per date (daily)
One column per entity attributes:
Price/Book <- a value for GE
Price/Earnings <- a value for GE
GICS Sector < - ID to a GICS Sector name table (which is not counted in my 33,000 table count)
...etc. to the N'th data item/column for the Value group

tblIBM-Growth <-company/entity specific "IBM" and group specific "Growth" group table
One row per date (daily)
One column per entity attributes:
Earnings Growth YoY <- a value for IBM
Revenue Growth YoY <- a value for IBM
GICS Sector < - ID to a GICS Sector name table (which is not counted in my 33,000 table count)
...etc. to the N'th data item/column for the Growth group

tblGE-Growth <-company/entity specific "GE" and group specific "Growth" group table
One row per date (daily)
One column per entity attributes:
Earnings Growth YoY <- a value for GE
Revenue Growth YoY <- a value for GE
GICS Sector < - ID to a GICS Sector name table (which is not counted in my 33,000 table count)
...etc. to the N'th data item/column for the Growth group


GolferGuy & skhanal - Are you suggesting putting all of the data from all tickers in one table per group (Value, Growth, etc.)? If so I would have 3300 new rows/records every day and that table would also be 1300 fields wide. Remember this is a daily time series so for every ticker I have one data point to store every day for every data item. That would make for some massive tables.

skhanal - I have a few colleagues advising. So far the feedback I am getting is that I just have to find ways to size things down or pony up for a huge operation/database. Size down is the only option right now. Still considering what to do exactly....