Crontab with Selenium and Chrome Driver

This is how to setup a crontab with Selenium and Chrome Driver.

Setup

  • Linux Ubuntu 16.04 with GNOME
  • Vim
  • Python 2.7.12
  • Bash 4.3.46

If you are here I assume you have some knowledge of cron. Otherwise read this cronjob tutorial.

Let’s pretend that we have a Python script in this directory:

~/Documents/scripts/awesome.py

Just for reference. You should use this for troubleshooting purposes:

Review the system log file here: /var/log/syslog

Send the stderr and stdout to a file like this. Right after the last crontab command:

> awesome.log 2>&1

Read more about the redirection syntax here.

Review your Python script

To be semantically correct, your header should have this:

#!/usr/bin/env python

Edit your crontab

Edit your crontab with crontab -e

In this example you run a job every day at 7:45am.

45 7 * * * export DISPLAY=:0 && cd /home/your-username/Documents/scripts/ && /usr/bin/python awesome.py > awesome.log 2>&1

To learn what 45 7 * * * means. Go to this tutorial about automating tasks with crontab.

Since cron runs with a limited number of environment variables. You need to set the DISPLAY variable.

This explains more about the DISPLAY variable.

DISPLAY consists of a keyboard, a mouse and a screen. The display is managed by the X server.

If I want to open Sublime in Linux. X server is the framework that helps me display the program into a window. If I move the mouse then it helps me go to an exact pixel in the screen.

Lookup more details on Wikipedia as X Window System.

The X server receives input from the keyboard and mouse and displays to a screen. A web browser and a terminal run on the user’s workstation.

If you do this on your terminal:

echo $DISPLAY

Most likely you will get:

:0

Which means that DISPLAY variable is set to :0.

If you do this on your terminal:

env | grep 'DISPLAY'

You will get:

DISPLAY:=0

Other examples of DISPLAY could be:

  • DISPLAY=localhost:4
  • DISPLAY=google.com:0

The syntax for DISPLAY is hostname:D.S which means screen S on display D of host hostname.

hostname:D.S

  • hostname is the computer name where the X server is running. If there is no hostname then it means the localhost
  • D is a sequence number. Usually 0. It can be different if the computer has many displays connected.
  • S is the screen number. 0 is the default.

When you echo $DISPLAY. The hostname is omitted and it shows only the default DISPLAY number: :0.

Crontab and DISPLAY

Selenium opens the browser if you are using the Chromedriver.

Cron doesn’t have the DISPLAY environment variable set.

You need to set the DISPLAY variable inside the crontab such as:

export DISPLAY=:0

Going back the example where you run a job every day at 7:45am.

45 7 * * * export DISPLAY=:0 && cd /home/your-username/Documents/scripts/ && /usr/bin/python awesome.py > awesome.log 2>&1

You are setting the DISPLAY variable before executing any command.

You are using the full path to where the files are.

And you are sending stdout and stderr to a log file for troubleshooting.

Related Post

Go to the Previous Directory in Vim I use Vim.Vim is the most productive text editor if you know some vim commandsWhen I open a file and I am done with it. I often want to go...
Export Data From Firebird Database to CSV File Background Some software applications use a database called Firebird. This is an open source database. If the software application is compiled you ca...
Installing Ruby on Ubuntu Installing Ruby on UbuntuAs seen on RVM...Go to the terminal in Ubuntu.Add the public key from RVM$ gpg --keyserver hkp://keys.gnupg.n...
Installing Hipchat on Ubuntu Tested on:Ubuntu 12.04 LTS Thinkpad T400, 2GB RAM Intel Core 2 Duo CPU P8600, 32bitSources:Official hipchat linux instructions Ask U...

Leave a Reply