There are a lot of reasons, one of the big ones being that the Republicans gutted the funding which forced Health & Human Services to scrounge for money which forced them to use almost 50 contractors to build this thing. HHS screwed up in two major ways. First, they didn't appoint a top dog contractor who was responsible for integrating this mess. HHS thought they had the in-house expertise to handle managing it all, and they clearly did not. They also chose a Canadian shop that had a track history of poor performance on government contracts. I have no idea how they got the contract, there's talk that there was some higher level quid pro quo going on, in that the company that bought out the main contractor is owned by Obama fund raisers and that fundraiser-owned company has been hired to fix the mess.
But the second one was a biggie: HHS specified that the system would run on top of a database system called MarkLogic. Not Oracle, not SQL Server, not IBM bringing in something. Most databases used in business for record-keeping use SQL, Structured Query Language. This organizes data in to tables that control the amount of redundant data and do lots of things to enforce data quality. SQL is predicated on algebraic set logic where the tables (sets) are manipulated using set operations: intersections, joins, differences, etc. I've been working with relational databases for about 30 years and I absolutely love the way it works. It's extremely fast, it's extremely robust, and has a very long and proven track record.
MarkLogic is not an SQL database. It uses a technology called NoSQL, which eschews a lot of the strengths of SQL databases, gaining speed, a smaller footprint in the OS, and lower cost. It is a datastore, but not like SQL. It has such a small footprint, as a matter of fact, that pretty much all smartphones use it internally to structure operations.
And MarkLogic seems to be a pretty good database (I've never worked with it). The London Olympics last year used MarkLogic to organize and stream all their data and video, and it performed marvelously. There are lots of examples of MarkLogic performing well.
There's only one big problem: the developers had never worked with it before. So in addition to creating a complex system, the developers had to abandon a lot of their previously honed techniques and learn new ones. New development techniques, new testing techniques. The developers balked at the spec, but HHS held firm.
And now we have the mess that we have.
http://www.nytimes.com/2013/11/23/us/politics/tension-and-woes-before-health-website-crash.htmlhttp://beta.slashdot.org/story/194761