Databases

[sofer]

I know we didn't spend a lot of time with LevelDB on FAC6, but has anyone who has had a chance to play with it got any views on whether it might be a good introduction to databases in Node?

It's not so much the LevelDB database itself that interests me as the LevelUp wrapper for Node and the various level bindings. It looks like LevelUp could be a great learning route into databases in general without us forcing people down a particular database route (other than one or other key-value store).

See e.g.https://youtu.be/C-SbXvXi7Og

@nelsonic? @rjmk?

[rjmk]

Agree that levelup is very cool and also level in general makes sense as a first db. I think leveldb's modularity is the strongest argument for it, but there are other reasons as well. For example, being able to npm install the relevant parts definitely lowers the barriers to entry. Learning to struggle with installing software / databases etc does have some value, but distracts from what already has to be learnt with databases.

Also, it would be remiss of me not to mention dat

[iteles]

Maybe we could have 2 weeks of databases? 1 week with levelDB as an entry-level DB (I haven't used it much myself but @nelsonic will have some experience to share I'm sure) and then another to explore others like redis and elasticsearch?

[rjmk]

@iteles I think 2 weeks of databases makes a lot of sense. Using the second week for something like redis or elasticsearch (haven't used it, but hear good things from you guys) could be cool. Equally, sticking with level and going deeper in any of the many directions level opens up could be cool (keyspace design, various cool dbs built on top of level, building a cool db on top of level ...)

[nelsonic]

Only saw this now. [#TooManyNotifications]
I think we need a general introduction to Data followed by an explanation of the benefits of both unstructured and structured data storage.
The biggest thing people need to consider is how they are going to query their data.
And this is difficult for in-experienced people to know up-front before they have built something...
We should focus on getting people to be pragmatic in their choice of database/store.
e.g. I would not use Redis to store financial transactions.
and where data and queries are fairly predictable/structured I would store it in an SQL database; specifically PostgreSQL.

I'm still a huge fan of _RethinkDB_ which has the benefit of structured queries and _real-time_ updates over websockets, however, I'm not "pushing" it on the FAC students because its not available for "free" on any service (e.g: to try it on a demo app on Heroku...)

To your question of whether people should learn LevelDB and LevelUp, it depends.
Some people will be curious about understanding data structures from first principals, while others simply want to get an app built as fast as possible. LevelDB is worth learning for the technically adventurous, especially for experimenting with _electron_ but I highly doubt the FAC students/graduates will ever use it instead of _Redis_ for their web app ...
On Heroku, a LevelDB-backed Database is in-memory but gets erased when the process re-starts, so its fine for "Toy" apps but not when data persistence is required... and given that I don't think the FAC students are going to be deploying to AWS during the course (where they have control over EBS and Processes) I think its of limited value in the context of building/deploying web apps... (sadly!)

As for ElasticSearch it serves a very specific purpose and for that purpose it's the best-in-class.
Whether people in FAC end up building web apps/products that involve "_Search & Analytics_" remains to be seen. Maybe @SimonLab and @heron2014 can give some insight into their experience of using ElasticSearch for the Contact Management app...
There is obviously a learning-curve with ElasticSearch, but in a crowded marketplace with little to distinguish "Junior" developers knowing how to implement a Search Engine can be quite a differentiator...

_IMO_ these should be the _FAC Four_:

• Redis - fast/simple/general-purpose/pub-sub
• PostgreSQL - Transactions & structured/predictable data.
• MongoDB - relatively flexible JSON document store with good query syntax
• ElasticSearch - Document Storage, Search & Analytics. The Search Engine used by Netflix, Github, The Guardian, Uber, Facebook, NYTimes, SoundCloud, Trello, StackOverflow, Wikipedia, etc..!

And then we should tell people to find one other DB over the weekend and present its' use-case, pros & cons to the rest of the cohort.

Thoughts?

[sofer]

I agree with @iteles on two weeks for databases.

The problem I have with the idea of a _FAC Four_ is that I don't think we have enough time to cover them all adequately. If we are going to offer a general introduction to Data (and I agree wholeheartedly with @nelsonic on this one) then I am not sure we have enough time for even two databases. Let me suggest a _FAC One_.

So, if we were going to introduce just one database, which one would it be?

[sofer]

(@rjmk "The high level goal of the dat project is to build a streaming interface between every database and file storage backend in the world." This looks fantastic.)

[rjmk]

I'd forgotten about the irritating ephemeral file system on Heroku. I think that's what stopped @rub1e level evangelism during FAC 5.

'Introduction to Data' 👍. I would suggest that it is interleaved with the course though. A small amount before using databases, and more after, looking at it from different angles and with different levels of knowledge in the classroom.

Given what people in the space know best, sticking with redis might make sense.

MEAN is still a popular stack as far as I can tell, but seeing as we don't use express or angular, that doesn't say much about us using Mongo. I think it would be more forgiving than SQL while having some of the same flavour with writing queries.

Postgres has the advantage that SQL is used everywhere. And data normalisation is cool. There could be a lot to learn / teach in terms of data with postgres as a starting point.

Overall, I think redis, but it's not an opinion I hold particularly strongly

[sofer]

I cannot help noticing that some of the FAC6 project teams are now using Firebase and, of course, they are loving it--because it is so easy. There are competing priorities here: what's a good foundation for students' understanding of a topic vs what's going to be expedient for their projects vs what skills are likely to be useful on dwyl projects.

[tormod17]

We used Firebase for our Boat Launch project and it was fairly easy to use. It's hosted online so you don't need to run a db server, you can access it from the front end if you wish with just a script tag and a callback function. It's similar to Redis in that it uses key value pairs, this requires some thought into how you want to structure your data as every Hash requires a unique key (could use time stamp). Theodashboard or GUI is really simple you can edit import (JSON format) and export data fairly readily. One thing I did notice was if you had a large data set I couldn't see how you could search/ query easily from the GUI. There are some cool features like GeoFire which allows you to access snapshots of the database via sockets for real time updates and plot co-ordinates onto a map.
Here is a Stack Flow link with pros and cons

[hdrdavies]

TeamMunch are currently using Redis. We went for it as it's super easy with node and that it's fast & flexible. I'd say that if you were going to do a FAC _One_, as discussed above, I would recommend Redis as the one to go for (even though I really enjoyed using Firebase). Most teams would likely be using it in their MVPs so equipping them with the appropriate knowledge seems to make sense.

[naazy]

I agree with @hrdavies. Firebase is incredibly easy but perhaps too easy and it was definitely good and useful to learn how to use Redis first (and to incorporate it with Sockets). We decided to use Firebase for Skipit because the client wants to be able to see all of the user data on a dashboard. It's really cool because it’s so user friendly and is real-time but it's only free for upto 100 connections at once so maybe not great for bigger apps (unless you have lots of dollah billz $$$$ to burn through). If you want more connections then it costs at least $49/month..

On 23 Jan 2016, at 14:27, Huw Davies notifications@github.com wrote:

TeamMunch are currently using Redis. We went for it as it's super easy with node and that it's fast & flexible. I'd say that if you were going to do a FAC One, as discussed above, I would recommend Redis as the one to go for (even though I really enjoyed using Firebase). Most teams would likely be using it in their MVPs so equipping them with the appropriate knowledge seems to make sense.

—
Reply to this email directly or view it on GitHub #17 (comment).