Results 1 to 3 of 3

Thread: A new kind of database

  1. #1
    Join Date
    Nov 2020
    Location
    Utah
    Posts
    6

    A new kind of database

    As everyone knows, there are lots of different databases that have been invented over the years. I have invented another one and anyone is free to download it and try it out. It is built on top of an object store of intelligent data objects called 'Didgets' (short for data widgets).

    Highlights:
    1) Each relational table is a columnar store where each column is basically a highly optimized key-value store so the table has a very flexible schema. New columns can be added at any time or existing columns can be deleted or modified. Queries that only address a limited number of columns in the table only need to read in those columns from disk.

    2) Fast queries without needing separate indexes. In my testing, queries execute about 4x faster than the same data imported into Postgres, MySql, or SQL Server. There are no indexes to maintain and updates do not require changes to both a table and its index.

    3) Three dimensional tables. Since each column is a separate key-value store, each row can have multiple values mapped to its key for each column. So importing a Json file with an array for 'Phone Number' instead of just a single value works just fine. A Json 'document' with 4 phone numbers will be represented in the 3D table as a row with 4 separate values or a 'depth' of 4 for that column.

    4) Fast pivot tables. Pivoting tables with millions of rows only takes a few seconds.

    5) Interactive tables. Just double-click on a table to do a 'SELECT *' query. Just double-click a value in one of the rows to instantly see all the rows that have that same value. Click on a column header to instantly see all the unique values in the column and how many rows each value has.

    I just started the beta test. It won't do everything that other databases can do yet (e.g. not all JOIN operations are supported), but lots of features work. I have tested it out so far with tables up to 2500 columns and up to 120 million rows.

    www.Didgets.com is the website where you can get more info and see demo videos.

  2. #2
    Join Date
    Jan 2021
    Posts
    8
    Number of tables that can be read is dependent on the software used. I can't understand point number 1 properly

  3. #3
    Join Date
    Nov 2020
    Location
    Utah
    Posts
    6
    Many relational database systems are 'row oriented' meaning that all data within a row is stored together. If a particular query cannot be satisfied by just using indexes on the table, the system must perform a 'table scan' meaning that all the data in the table must be read in from disk to memory. If you have a fairly large table (e.g. millions of rows, hundreds of columns), this can be a lot of data.

    For example, let's say you have a 200 column table but you only need the data from a few columns (e.g. SELECT name, address FROM <table> WHERE name LIKE 'A%', if you don't have an index on the name column (or almost all the names start with the letter 'A') then all the data in the table must be read in from disk.

    With the Didgets system, all the data in each column is stored separately so only those columns needed to satisfy the query must be read in from disk. In the example above, only the names and addresses would be needed. The other 198 columns worth of data could remain on the disk. For some queries, this capability could make a huge difference.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •