Ultra-orthodox jews study a lot. One of the most revered occupations one can have is being a student in a Yeshiva. In fact, if you are a Torah pundit, you can get married easily, with girls coming from rich families, even if your parents are poor. Kind of the Jewish Revenge Of The Nerds.
There are two types of books that they study: the Torah – meaning, first five books of the bible, and then there’s the Talmud, the Oral Law, that has passed through generations of Talmudic Law researchers, which is what these Yeshiva guys really are. Actually this Oral Law is the main substrate to the Orthodox stream in Judaism.
The knowledge management world has just recently caught up on Oral Law. If you are learning something, and are presented with a document and video, most chances that you’ll go for the video. It is so much better to have someone explain something to you, than try to figure it out by yourself. More engaging, more reassuring, more natural – and that’s why video – from how-to tutorials to online academic courses – has surged dramatically in recent years.
Korra is the newest search engine on the block that we built with the aim of searching both of these Torahs – videos and documents. It can ingest millions of such files and spit out an answer from a specific second in a video or a certain paragraph inside a document. How does one go about building such a big-data monster?
For all the years I have been a developer, the most interesting part was not the speed of the CPUs or the minimization of the computers, but the size of the systems that teams can build. With this growing size, complexity and potential pitfalls grew as well. And with that, the underlying methodologies evolved – changing so much in front of our eyes. Evolution of thought.
Much of this evolution is done by hands-on practice. Things you can’t learn from textbooks, much like you can’t learn swimming by watching YouTube videos: you have to have someone teaching you.
I guess the only way to learn it is either from experienced team leaders, or by reading articles like this one. So, in the coming articles we are going to describe the Oral Law of large knowledge portal development – what’s going on behind the scenes at Korra.ai.