In the midst of our random data exploration, Laure and I started playing around with Hadley’s movies dataset and noticed that there were a lot of old cartoon animations… I mean REALLY old. So we got excited and wondered if we could find Youtube links for all these old animations. Indeed we could! Here’s how we did it. As always, the full analysis is on github.

In R, Load in ggplot2movies::movies, which has something like 60 000 movies
Reduce this dataset to animations of only 10 minutes or less and then arrange by year and descending rating, then select only ~~1 best animation per year~~. Hrm… actually, select each year’s 3 best-rated cartoons. You’ll see why later. This reduced dataset has <300 rows… much more manageable. Of this, only keep the title and year, you don’t need the rest.
Now let’s create a column that will give us good search results… in order to do that, prepend with cartoon and put the year between parentheses so that each row looks like this: “Cartoon ”It’s the Cat” (2004)” (the backslashes escape the quotes inside the string… dontworriboutem).
Now feed each one of these lines into bing.com (google/yahoo don’t let us!) and capture the results using httr::GET()
Now we have the whole webpage with the results inside. We use XPath to try our best to grab only the parts of the webpage that we want, namely the search results. Find the first Youtube hit. It takes a LOT of cleaning to figure out what you need and what needs to be thrown out. From this, grab the title of each search result page and the link (by the way, at this point, we have a shorter list because not every movie has a Youtube link).
Now we have to face the reality that the search result may have given us a Youtube link that wasn’t the film we wanted. In order to understand whether we did or not, we use a package called stringdistance. This measures the difference between the title we were looking for and the title of the Youtube hit we got. Sometimes you look for “Unsteady Chough, The” and get back “The unsteady chorough”. So for this analysis, I found the _qgram _method to perform best. Unfortunately, sometimes even the string distance alone isn’t a good predictor. One or two misspellings in a film with 3 letters is a bigger deal than 5 misspellings in a film with 30 letters. Therefore:
Come up with a Percent parameter where you divide the stringdistance() by the number of characters in the title you were looking for. This will give you a good indication of how much error there is per string.
Based on an evaluation of the string distances and percents, we realized that very few cartoons have good correspondence between the title looked for and the title found.This is why one movie per year doesn’t work and we had to go back and select the 3 best cartoons.
After analyzing the results carefully, we decided to cut the dataset to only stringdistance() <8 and Perc <100 (although results vary a lot here). This left about a hundred cartoons.
So now that you have good cartoons and their Youtube links, put some html code in front and in back, and use cat() to push it all out into an html file!

Final thoughts

We tried to find the images for each video entice viewers to watch the cartoon, but this was reeeeeally hard to do! All the search engines we found push the images as data or in iframes so we couldn’t capture the image with GET. We finally gave up 🙁
We could have probably found more hits if we had looked at more movie sites than youtube, but we didn’t like the idea of going to whatever site… who knows what ads they might have popup or whatever.
It’s lots of fun to see how cartoons have changed through the ages! It’s amazing to see what they could already do in 1922 and how quickly cartoons improved over time! Click around and see for yourself!
Some of the early attempts were full of errors, but in a way it feels a bit psychedelic to click on these links… it’s Youtube roulette! I dare you! Click around »here«.