Mark Venn, BSc Web Technologies
* commended *
This research project addresses two questions: Are today’s databases fit for purpose? Do we need to shift to a new way of storing and accessing the new phenomena of “big data”?
By using SQL and NoSQL database technologies I have attempted to look at the ways in which “big data” is stored and used. Scientists at CERN are working through 800 trillion particle collisions, at a rate of approximately 1 petabyte a day. A Boeing 747, on a single crossing of the Atlantic, produces around 640 terabytes of data. This data needs to be stored, and importantly, accessed at a suitable speed. How long does it take to find a particular record in 1 petabyte of stored data? If it is stored in a different format, can this time be reduced? By using the same data in both technologies I am trying to find if one solution is better than another for different operations.