Setting up twitter streamR Service on an Ubuntu server

I am working on a super-secret project for which I am harvesting a highly confidential source of data: twitter ūüôā The idea is to gather a small amount of twitter data, but for a long time… maybe a year. I tried to use the package TwitteR, but it can only¬† grab up to a week of tweets… it’s not really… Read more →

Announcing ShinyTester – a package that helps you build Shiny apps

Shiny is awesome, but can be a bit daunting and easy to make mistakes in.¬†I recently came back to Shiny after a hiatus of a few years and it was much more challenging than I feel comfortable admitting. I was making bonehead mistakes¬†like writing something instead of output$something, confusing where to put Output commands vs Render commands, etc. I would… Read more →

Pop quiz! What is this chart saying?

I have been reading more and more about how people can’t¬†interpret charts… which kinda never occurred to me, if I’m gonna be very honest. ¬†Anyway, it kind of made me think of actually testing people informally, to see for myself. So I’ve been doing just that: showing colleagues, friends, etc a chart that we created interactively during the first Accra… Read more →

Heart-shaped wordcloud, celebrating Colombia peace treaty

This is a lightening quick post just providing the script to draw a heart-shaped wordcloud, using the awesome wordcloud2 package. See the resulting image here:   Apparently, the original code allows you to fit a wordcloud to any shape, even custom shapes, but I didn’t find that functionality pushed out into R yet (a peace sign would have been awesome… Read more →

Track your local R scheduled tasks with CommandCenter2000!!!

There are many ways to automate your scripts running, for example using RScript, or in-R itself¬†(and now even as an add-in for RStudio) (check all here). But after a while, it can get a bit overwhelming to track tasks and ensure they are firing as planned. In order to address this, I have developed some futuristic¬†advanced tech that lets us… Read more →

I’m leading the Accra R-User Meetups!

I am not sure how, but I found Data Science Africa, whose mission it is to promote data science in Africa. For now, they are limited to organizing and running meetups across Africa, mostly focused on R, although there are higher albeit more ambitious targets. Anyway, since I’m in Accra, I contacted them to see if I could help in… Read more →

Make dashboard Tiles/Notecards in R

Update!! Due to the popularity of this script, I released it as a package. To install it, use: devtools::install_github(“mexindian/TileMaker”) ———————————————————– Dashboard Tiles/Notecards are a great way to visualize just one number, and make it big and pretty. They can emphasize results in an easily digestible and colourful format. I was surprised that there was no way to create tiles offline… Read more →

KPI dashboard in R with animated icons

So Key Performance Indicators (KPIs) are all the rage in the dashboarding community…¬†well everywhere really. The premise is simple… check a list of measurements against targets and show how they compare using some kind of visualization. I haven’t yet seen, however, a version that can utilize animated icons to display indicators that REALLY need attention. So here you go, a… Read more →

I’m proudly featured on r-bloggers!

I am proud to announce that with a little bit of encouraging from friends, I submitted my blog for inclusion in the prestigious r-bloggers and it was accepted! r-bloggers is the biggest roll up for high quality r related blogs and articles. It is actively maintained and thus is a great place to keep visiting to keep your finger on… Read more →

R Tagosphere!

This post explores the inter-relationships of¬†StackOverflow Tags for R-related questions. So I grabbed all the questions tagged with “r”, took the other tags in each¬†question and made some network charts that show how often each tag is seen with the other tags. The point is to see the empirical relationships that develop as people organically describe their problems with R.¬†Full… Read more →

10 things you didn’t know about taxi costs in Accra

  “… but I paid 10 yesterday!” Having just arrived in Accra, Amit and I were trying to wrap our heads around taxi prices. We knew that we had to negotiate with the taxi driver¬†but often the agreed-upon price kinda didn’t make sense…at all. Our first hypothesis was that the price was completely random! Then we noticed we tended to… Read more →

How to move ODBC DSN information from one computer to another

YAAAAAAAAY!!!! I got a new computer!!! Oh crap how do I move the DSN connections that I made on this computer?? This is how (assuming you’re on windows)! But before, we do, we have to make sure we have the right drivers… so lemme first go over the basikses my precious: TO ADD NEW CONNECTIONS: Build a shortcut on your… Read more →

Clickable list of the best animations since 1900, gathered the geek way.

In the midst of our random data¬†exploration, Laure and I¬†started playing around with Hadley’s movies dataset and noticed that there were a lot of old cartoon animations… I mean REALLY old. So we¬†got excited and wondered if we could find Youtube links for all these old animations. Indeed we could! Here’s how we did it. As always, the full analysis… Read more →

Locations for 75000 dams

The last task I performed for AQUASTAT was to try to find the best way to estimate the anthropogenic¬†evaporation from dams. There will be a paper coming out soon, but in the meantime, I provide one of the fun outputs, a map of 75000 dams! ok okok… it’s not REALLY 75000 dams, this is an duplicate-containing amalgam of: AQUASTAT dams… Read more →

Chick flicks vs action movies!

I co-wrote this analysis with Laure Belotti¬†to identify the difference between movie scripts for chick flicks and action movies using sentiment analysis, among other stuff. Rather than creating¬†a blog post, we¬†made¬†a freestanding webpage because I wanted to play around w/ bootstrap and make it pretty and stuff. Anyway, the analysis is here:¬† Leave comments below! Read more →

Sentiment analysis on my girlfriend’s text messages

When I told my friends that I wanted to give my girlfriend an infographic of us (centered around a sentimental analysis of our texts) as a gift for our first anniversary, most of them told me that was a terrible idea. Yeah… well… CHALLENGE ACCEPTED!! Without further ado, this is what love looks like: What… um…. what are we looking… Read more →

Excel is how I do maps…

Following the footsteps of this cool post I saw by¬†JohnNelsonIDV, I wanted to try to and make a map using Excel. Fast forward 2 days and the opportunity came up. Lo and behold, a Map of Reference Evapotranspiration built in Excel! Instructions: Read data file into Excel (skip top rows with metadata), make sure to delimit correctly so that each… Read more →

Irrigated Crop Calendar Vizualisation

Visualizations made for FAO AQUASTAT: After individual Irrigated Crop Calendars for all countries were combined, it allowed for some interesting analysis of global irrigation. These included “The Big Picture” (at two levels of resolution), as well as viewing irrigation by region and by crop. All explanations and visualizations on:¬† Read more →

Simplified renewable water resources, visualized

A visualization prepared for FAO AQUASTAT: In an interest to simplify the information about renewable water resources in all countries in the world, we prepared simplified visualizations of water flows through a country. Please navigate to the AQUASTAT Water Resources page¬†and use the “Figures” dropdown to see the flow for each country (as well as the definitions of the abbreviations. Read more →

The Reddit effect…

So I had a happy little blog… no one really paid attention, but I didn’t much care because I was happy. And then I posted one of my visualizations to reddit. This is what happened: (None of the log¬†analyzers¬†I found would make a plot like this so I made it myself by parsing the log file itself,¬†script¬†is on github). So… Read more →

Irrigation area treemaps

These visualizations were performed for FAO AQUASTAT. These treemap charts allow for hierarchical organization of information of different types. In the image above, for example, the irrigation areas for each country are represented as the big boxes, within which the technology utilization is displayed in different colors. This allows users to see a large amount of information quickly. View more… Read more →

Transboundary surface water flow

A visualization generated¬†for AQUASTAT of FAO. Two types of charts¬†were prepared:¬†Sankey and Network. The Sankey plots allow for an ‘automatic sorting’ of countries based on who is the water tower and who is the water source. This visualization is useful to demonstrate where a country falls on this continuum. The network plot shows the¬†automatic arrangement of countries with¬†their ‘water-neighbours’¬†without being… Read more →

Chord progressions of 5 000 songs!

Update: Full analysis and everything you need at my github The database contains analyses of over 5000 songs*. These analyses are uploaded by users and allow for all these songs to be analyzed in bulk, as well as individually. One of these ‘all song’ analyses enables users to gather chord progressions on ALL songs (see the analysis file… Read more →

Best way to make a custom palette with ggplot2

      A better way to pick colours with ggplot2 First, get the excellent package by Marco D Visser: library(devtools) install_github(“MarcoDVisser/choosecolor”) call it library(choosecolor) This lets us interactively pick colors ourselves. Just for fun and to get the hang of it, experiment with color.choose() now make a palette. In the syntax below means that you get to pick 4… Read more →

Excel basics

Most of the things posted on this page will make reference to Visual Basic for Applications (VBA). How do you get to the Visual Basic Editor? Excel versions prior to 2007: Under the¬†Tools¬†menu, go to¬†Macros, and the option should be there. Excel 2007: Available through the¬†Developer¬†tab on the ribbon. If you don’t see a developer tab: Add the Developer tab… Read more →

Cover images for facebook, google+, whatever

So I guess everybody let’s you upload a “cover picture”…. something that should be beautiful, emblematic of you, deep, thoughtful etc. Well… I found a bunch of my pics that fit the wide format nicely. If you find one you want: Click the picture so that you get the high-rez version. Right click + save image Upload on whatever social… Read more →

Arduino theremin concert

One of the arduino tutorials was for a theremin. With a few modifications I was able to properly express my sophisticated musical abilities. Enjoy! Arduino code follows (prolly doesn’t make TOO much sense without the wiring schematic… if there’s interest I’ll put up a picture). int sensorValue; int sensorLow=1023; int sensorHigh=0; const int ledPin=13; void setup(){ pinMode(ledPin,OUTPUT); digitalWrite(ledPin,HIGH); Serial.begin(9600); while(millis()<5000){… Read more →

%d bloggers like this: