5 min read

Track your local R scheduled tasks with CommandCenter2000!!!

2016-05-29

There are many ways to automate your scripts running, for example using RScript, or in-R itself (and now even as an add-in for RStudio) (check all here). But after a while, it can get a bit overwhelming to track tasks and ensure they are firing as planned. In order to address this, I have developed some futuristic advanced tech that lets us do that. I hereby present to you: the CommandCenter2000!!! Yes, it’s Y2K compliant! This is what it looks like:

 

Capture

 

It has the following features:

  • It’ll tell you when the script ran last
  • whether it was successful or not (green/red light)
  • and will help you rerun failed scripts in 2 ways.

How to make your own:

diagram

(Follow the steps in the schematic above: red for setup, and yellow to run it. These are explained below)

  1. Setup the folders where there are running tasks. Concretely, inside the working folder for each script, you need a .bat file that will run that specific script. Within the bat file*, the code will look like this for example:

    @echo off<br /> “C:\Program Files\R\R-3.2.2\bin\x64\R.exe” –vanilla –slave CMD BATCH “D:\Project X\analyzer.R”<br />

  2. OK, now you need to schedule the task in Windows (I like scheduling it myself so that I can set some specific requirements… in particular, I want tasks to fire only if connected to certain wifi). Don’t know how to do this? [There are many ways to automate your scripts running, for example using RScript, or in-R itself (and now even as an add-in for RStudio) (check all here). But after a while, it can get a bit overwhelming to track tasks and ensure they are firing as planned. In order to address this, I have developed some futuristic advanced tech that lets us do that. I hereby present to you: the CommandCenter2000!!! Yes, it’s Y2K compliant! This is what it looks like:

 

Capture

 

It has the following features:

  • It’ll tell you when the script ran last
  • whether it was successful or not (green/red light)
  • and will help you rerun failed scripts in 2 ways.

How to make your own:

diagram

(Follow the steps in the schematic above: red for setup, and yellow to run it. These are explained below)

  1. Setup the folders where there are running tasks. Concretely, inside the working folder for each script, you need a .bat file that will run that specific script. Within the bat file*, the code will look like this for example:

    @echo off<br /> “C:\Program Files\R\R-3.2.2\bin\x64\R.exe” –vanilla –slave CMD BATCH “D:\Project X\analyzer.R”<br />

  2. OK, now you need to schedule the task in Windows (I like scheduling it myself so that I can set some specific requirements… in particular, I want tasks to fire only if connected to certain wifi). Don’t know how to do this?]6 (that’s how I learned)!

  3. OK, now tasks will be firing off! Now you need to get the command Center script from my github account. Go it now!

    1. This contains:
      1. the CommandCenter R script
      2. a .bat file that will run the CommandCenter2000!!! itself. I recommend creating a shortcut to this bat file on your desktop or whatever. That way you can run it whenever
      3. the html file that reports the status of each task.
      4. ignore the rest of the stuff
    2. Ok, now you have a local CommandCenter2000!!!.
      1. Set up the WORKING DIRECTORY to wherever you installed the folder.
      2. Now, for each script you’d like to track in the CommandCenter2000!!!, add a reference to that script in the main CommandCenter2000!!! script. By “reference”, I mean:
        1. Copy the block of code that looks like it should be copied. (hint, there’s START and END comments). Within each:
          1. Give that script a human name (this is what will be displayed)
          2. Specify the path and name of the R file (remember the whole “/” vs “&#8221; debacle for windows paths)
          3. Specify the path and name of  the .bat file
        2. That’s it! Rinse and repeat for every script that you would like on your CommandCenter2000!!!

  4. OK, setup is done. Now, whenever you want to check the status of scripts, you can just run your command center. Of course, you could run the CommandCenter2000!!! itself on a schedule… but putting something on a schedule to check whether other things are running on schedule correctly is a bit…… mind-bending. Careful you don’t break reality.

  5. Everytime you click it, you’ll get the html file coming up shown above. In detail:

    1. “thing“: is just the name of the analysis
    2. “lastRun“: is how long ago the analysis ran (in days… although obviously this is configurable).
    3. “status“: (this is the last column) has a red or green light to indicate whether the script ran successfully or not. If it didn’t, the red light will have  a mouseover effect whereby you’ll be able to tell what the error was…. which looks like this: ddThis mouseover effect will help you debug… using the following two options
    4. “SomethingWrong“: provides a link to the R file, in case the script failed because your R script is messed up. Don’t click the link, but right click and “copy link”, then open it from R (sorry for roundabout way, but unfortunately it’s very difficult to launch a local file from HTML for security)
    5. “runLink: provides a link to the bat file that will run the script (more on this later). Use this if  the script failed but you think the R script is correct and just needs to rerun (for example if the network crapped out), just click, download the bat file and run it… everything will rerun automatically.

 

So clearly, this is a very rough first-pass attempt to keep track of scheduled tasks. If you’d like to collaborate, I’d love to evaluate some pull-requests.

 

 

*WARNING: This approach uses .bat files, which could damage your computer or corrupt your data. Be VERY CAREFUL, and don’t trust anybody else’s bat files without first examining them in a text editor.