Big Data & Search Technology

March 9, 2012

Sébastien, Rémi & Jim: Three Guys Who Love the Big Challenge of Big Data!

I had the good fortune to share some one-on-one coffee breaks this week with the fellows who head up the Web Search & Analytics division in Exalead R&D: Jim Ferenczi, who is in charge of the Web Data Mining team dedicated to enhancing the Exalead Web platform; Rémi Landais, who leads the Innovative User Experience team, whose mission is to invent and discover next-generation applications of Big Data; and Sébastien Richard, who’s the big cheese of the division.

I met first with Jim, our platform guru. One of his charges is building the Exalead WWW search index. Though Exalead’s public WWW search engine is now the world’s third largest, behind Google and Microsoft’s Bing (now that Yahoo! has handed over the search reins to Bing), Jim’s mission is not, as one might think, to cast the widest possible net across the Web, but rather the smartest one.

For one thing, there is certainly no shortage of junk on the Web, and Jim relishes the challenge of developing an ever more discriminating crawl ecosystem. It’s not only an approach that yields higher quality results for users, it also helps to optimize the size of the index. And, as Exalead is not Microsoft or Google, this is a welcome economic advantage.

As Jim notes: "The financial realities of our start-up origins made the drive for the best, rather than simply the biggest, index a necessity. But the real economies come from our core focus on lean computing. From the beginning, we set out to process the largest possible volume of data at the fastest possible rate on the smallest possible number of commodity servers. As a result, our Web search engine operates at hardware economies-of-scale no other WWW engine can touch, and as our Web and enterprise platform are one and the same, it means we’ve been able to deliver massive linear scaling on a tiny, low-cost footprint to the enterprise as well."

As if all this weren’t enough to keep Jim busy, he has to continually advance the robustness and sophistication of our semantic processing pipeline while still pushing the boundaries of lean computing. It a crazy mission, but then who cares if you can ‘handle’ Big Data if you can’t make sense of it?

Next up in my Big Data series: Man-with-Crazy-Mission #2, Rémi Landais.

>> To comment or share, please got to the original post!