Has anyone heard of the Say 'No' to SQL movement?

[abhijit]

I was recently reading a blog (http://blogs.citytechinc.com/sjohnson/?p=89) where I came across this movement.


Here's a list of projects that could potentially replace RDBMS.

Project Voldemort

Ringo

Scalaris

Kai

Dynomite

MemcacheDB

ThruDB

CouchDB

Cassandra

HBase

Hypertable

[Shaggy Hiker]

Even the blog only went as far as saying that these techniques would supplement RDBMS, as there are situations where they are sub-optimal. Other than that, I found the blog to be kind of vague and airy, as if a True Believer is attempting to preach to a choir.

[abhijit]

With the amount of time that we have spent in learning and implementing RDBMS based solutions for every project, it would be kind of impossible to get rid of it. Oracle and SQL Server's market share is safe for now.

There's a computerworld article (http://www.computerworld.com/s/article/9135086/No_to_SQL_Anti_database_movement_gains_steam_?taxonomyId=173&pageNumber=2)covering this.




Don't call them databases. Amazon.com's CTO, Werner Vogels, refers to the company's influential Dynamo system as a "highly available key-value store." Google calls its BigTable, the other role model for many NoSQL adherents, a "distributed storage system for managing structured data."

They can blow through enormous amounts of data. Hypertable, an open-source column-based database modeled upon BigTable, is used by local search engine Zvents Inc. to write 1 billion cells of data per day, according to a presentation by Doug Judd (PDF document), a Zvents engineer.

Meanwhile BigTable, in conjunction with its sister technology, MapReduce, processes as much as 20 petabytes of data per day.

[techgnome]

Hmmm.... couldn't get to the content of the blog ... probably my firewall or something. No biggie. I googled some of the names in that list... I don't see how they compare to traditional DBMS... they seem to have a very different target in mind.... Of the six that I googled, 5 used used "Document Indexing" in its description. One had a result that explained that it uses BerkelyDB on the back end. Even still... based on the results I got back, I would agree they are not RDBMSs.... but that's like looking at a bowl of fruit and claiming that apples are dead because it isn't an orange. A document indexing system has significantly different requirements than a traditional RDBMS. Does that mean that RDBMSs are obsolete? I doubt it. I have tons of data that would not work as a document. conversely, there are millions of documents that can't be easily broken down into data points.

It's possible I've gotten the wrong gist of this, seeing as how I can't access the entry.

-tg

[techgnome]

*snort* for something that is supposed to be part of the "No to SQL" movement.... it sure looks like SQL to me ... http://www.hypertable.org/hql/select.html

There is no typing for data...
Is it just me, or is that a red flag?
http://code.google.com/p/hypertable/wiki/HowHypertableWorks

Sorry... still not sold.

-tg

[abhijit]

I am not sold, but the more I read on this, I certainly am fascinated.

A few years ago (2002), I had a chat with a project manager who was advocating the use of XML to store everything.

At the time, I mentioned it would be impossible to duplicate the ease and flexibility of a RDBMS system in a XML based data store and the only way to do it would be to use a RDBMS system to store the XML based data. ;)

It does appear that their movement is being noticed.

[techgnome]

I guess it depends on how it is presented. If it's represented as "death to SQL" ... then I think it does itself a disservice. If it's presented as an alternative, and as the right tool for the right jobs at the right time... then I could see greater adoption.

-tg

[dilettante]

Most programmers don't work on systems that require the performance levels these non-relational DBMSs really target.

The basic idea is that imposing a relational engine on top of indexed datastores can be rather inefficient. I don't think it's about SQL so much as relational overhead, but SQL is so ubiquitous as a DDL and DML it becomes a sort of poster child for the real issues.

It is possible to use a relational DBMS in ways that limit its impedence to high transaction volumes and its inflexibility. At that point SQL itself begins to get in the way though, and a cleaner interface between application logic and the datastore can offer advantages.

Even IBM still supports IMS for customers with high volume requirements, as an alternative to DB2 which serves the larger population of customers who can afford a simplified, more symbolic view of data.

I think it comes down to where your pain point is. The vast majority of line of business software development has to worry more about the speed and ease of development and maintenance. A smaller set of problems requires more emphasis on volume and response time because they operate at much higher levels of activity.

I think it's a little bit like deciding whether you can use a language like Cobol or C# or have to drop down to C/C++ or even assembly level languages to meet performance requirements. For most business problems it would be wasteful to drop to lower level languages because it raises development costs. The vast majority of problems out there can be solved quite handily using VB, Pascal, Java, Cobol, etc. using cheaper programmers and/or getting more programming done more quickly.

Almost all database needs can be met using an RDBMS in the same way, again using cheaper people and/or gaining higher database development productivity.

Railing against non-relational datastores is kind of pointless. A (bad) analogy might be to talk about how a semi tractor-trailor makes a lousy minivan. Lots of us can find uses for a versatile minivan and may never need a semi. That doesn't mean a semi doesn't have its place when moving bulk freight.

[Nightwalker83]

If they replace sql then I have just wasted 6 months learning that stuff for absolutely no reason. However, I don't think they will get rid of sql in a hurry because so many systems use it and from what I understand it is still the primary method taught in colleges and schools.

[techgnome]

6 months? Try the last 20 years! I don't think anyone has actually (in this thread) stated that SQL is dying or that it should die. I think dilettante raises a good and valid point. For most of us SQL handles the job quite nicely most of the time. How ever, there are cases, rare as they may be, they do exist, where SQL is inadequate. That is a case that I mentioned were SQL no longer becomes the right tool for the job. That's when you have to look at something else. That's also not to say that the two systems can't co-exist... use the non-relational data storage for high volume transactions. Then during a dwin cycle the data could be off loaded, scrubbed, processed and put into a relational system... if that's the desired effect. I have a hunch though that data that's running through such a high volume transaction base is more likely to end up in a data warehouse, which is anything BUT relational (consider the vast volume of data that Amazon.,com must be able to quickly deal with... and then they offer suggestions on top of that.)

SQL and Relational data stores aren't going anywhere. Neither are non-relational data stores. But I don't think they are mutually exclusive either. One is a hammer, the other a nail gun. Jobs are similar, but their means are completely different.

-tg

[Nightwalker83]

6 months?

Yes, it was part of my web development course.

[Shaggy Hiker]

Whenever something new comes along, there are plenty of people who get out in front, banging the drum, and extolling the new technique as 'better than what we have'. These cheerleaders get a bit carried away with the new technology and extrapolate its better performance in one area to better performance in ALL areas. It's just a new tool, and will work well in its niche, while working less well in other places.

[dilettante]

Actually it isn't a new tool so much as a renaissance of another way of modeling, managing, and manipulating data. Most of you are probably too young and/or too new to computing to remember the hierarchical and network databases that came before large memories, large hard drives, and fast processors made the RDBMS practical.

It's not so new in another way too, at least in terms of the WinNT platform. There is one of these that ships and installs in every contemporary version of Windows out there. Until recently almost nobody but Microsoft employed it for much though. It used to be called Jet Blue (as opposed to the Jet Red most of us are familiar with from Access and Classic VB). Now it is called Extensible Storage Engine for NT, or ESENT.

There is even a managed code wrapper for ESENT now, though I think it is an unofficial (i.e. .Net community) project.

[NeedSomeAnswers]

Most of you are probably too young and/or too new to computing to remember the hierarchical and network databases

No not really, i would say there are quite a few older developers on this site.

Anyone who has worked somewhere that uses a Novell network will probably have come across a hierarchical database whether they knew it or not, Novell still use it for there Ldap database.