Case Study: Aggregating Daily Content from 500+ Sites

September 13th, 2012 by Ken Elkabany

We know that PiCloud is primarily associated with scientific workloads, and high-performance computing. So it may be surprising to hear that many of our earliest adopters were in fact web companies who aggregate content from all over the web. In fact, the s1 (scraping) core was in direct response to the needs of this class of users.

Zinc.TV (Division of TiVo) signed up only a couple of months after our first release. They’ve built an awesome service they call your “Internet Television Dashboard.” It’s a central location where you can search for any TV show or movie, and it’ll show you all the online sources–both free and paid–that offer it. They even have a “leanback” mode for when you aren’t feeling particularly decisive.

Zinc.TV maintains its comprehensive show catalogue by regularly scraping over 500 web properties using PiCloud. They take advantage of PiCloud’s easy parallelism for aggregating and curating data, and our interoperability with other clouds; Zinc.TV stores all their data in their own databases hosted on Amazon Web Services directly from PiCloud.

Gary Rose, co-founder and head of R&D at Zinc.TV, had the following kind words to share:


“PiCloud has been an extremely valuable partner for the growth of Zinc. We estimate that it cuts our operational costs for managing the infrastructure by over 50%, and allows us to release new sites in a fraction of the time.”

Gary highlights what we agree to be the core benefit of PiCloud: drastically reducing the man hours that go into “behind the scenes” work such as creating a robust, scalable distributed processing system, thereby increasing the time for unique, interesting work. We’re popular with researchers because we automate clusters so they can focus on their science. The same principle applies to all types of technological endeavors reliant on computational power.

You can download the full Zinc.TV case study, or view it in your browser below.

Tags: , , ,

Categories: Case Study, Success Story

You can follow any responses to this entry through the RSS 2.0 feed.

Leave a Reply