About a decade ago, I tried to index the web via the Dewey Decimal System. I had a site laid out similar to Google, where you could browse sites continuously starting from a given call, but the DDS is proprietary, and those people hate anyone who uses their IP without a license. You can Google everyone they’ve shutdown – places that weren’t even libraries – for using anything similar to the DDS. I reached out to the group that manages the DDS, and was taken offline before my project even started.
With all the corporate BS lately, and people looking for alternative options, I thought I’d take a search engine old school, and we’d index like Usenet. Except with an XXX.XXX.XXX format.
My original project used sharded Redis, with append logs to disk. I chose it for its in-memory speed and key-value store. Did some calculations, and I’d have to have millions of records just to consume my entire system’s memory.
I had a lot of plans for this before getting shut down.
Now that I’m older, I’m curious if I should be using MongoDB.
What are the benefits and drawbacks of each? Which would you use? And why?
Postgres is usually the DB to use unless you have a specific reason to use something else. Something like Cassandra or scylla might be good for your use case but I’d probably still start with postgres and evaluate a switch to something else if and when you need to
I absolutely detest SQL. I will probably go to my deathbed having never typed a line of SQL in my life. I don’t know why I hate it as much as I do, but I refuse to learn it – my brain won’t let me.
I’ve programmed in C, C++, Python, BASIC, JavaScript, Assembly, etc. I wrote CGIs for Apache in all these languages, so I didn’t have to touch PHP. Every time I get near SQL or PHP, I start wanting to pull my hair out.
sounds like you’re scared of it maybe
How can you hate it if you haven’t learned it? You don’t even know what it is.
Like when my toddler says he hates a food that he has never tried.
You’re missing out. Postgres is the gold standard for databases.
Oh, so you’re insane
You could say that.
In that case use MongoDB.
I wrote my own key-value database (with drive and memory storage), so I didn’t have to use SQL.
https://github.com/ki4jgt/PPD/
Just got bored one night and built a blog off of it.
Edit: I’m literally tempted to write an API server around this, add sharding, and call it a night.
I’m sorry to hear about your condition.
I get it. SQL is such an obtuse language. There are no shortage of programming interfaces to it because it is so dumb.
Now, do you hate SQL or RDBMS? Do the concepts of normalization (deduplication of data), primary and foreign keys, etc. bother you? (Not throwing shade, I hate matrix math, we don’t control what we like.) I ask because there are alternatives to not dealing with SQL but maintaining the benefits of a RDBMS - namely data integrity checks, atomic operations, etc. etc.
Because if you do all the programming necessary to guarantee data integrity, and all that good stuff on top of a key-value DB… well, you’ve just created an RDBMS from first principles.
It’s your passion project, use what makes you enjoy it. Just try not to fall into the trap of reinventing the wheel. It’s easier to use an RDBMS for it at first and switching to a more specialized tool later when requirements are clearer than the other way around.
An alternative is finding someone to partner up with and be your DB developer and DBA. Easier said than done, but it’s a great alternative.
SQL is certainly a different way of thinking, unlike any other language. I can see why it’s not for everyone.
And not just thinking, it actively imposes an architecture onto your codebase. It pretty much forces a CRUD API, which only really works well for a client-server structure. And it forces you to break up your data structures and introduce IDs to a degree that you would simply not do while programming normally.
I mean, I can see the appeal. If these constraints are fine for you and you don’t have other constraints, like sparse data, then building your whole application on top of SQL gives you clear answers for how to architect that.
But yeah, I also really resent this idea that it should be the default, because when it does not match the problem domain, you spend a lot of time working around the architecture that it imposes.
You can use SQL however you want
You don’t have to normalize your data. In fact, denormalization based on access patterns is a common optimization technique.
You can go very minimal on the sql in postgres if you choose. You can mimic mongo’s basic function using the jsonb columns. So you’ll still be inserting data. But that will be indexed json. Here’s a docs link. Oh and I also vote mongo
So this, then: DocumentDB