Category Archives: dataBites

Building a Coconut Coir/Soil Moisture Sensor with SMS:

I’ve been wanting to build this for a long time. We have a pretty extensive gardening project going on, and although we’ve installed drip irrigation we’re still using a lot of water. We’re testing a couple of ways to save water, including drip irrigation, coconut coir for soil moisture retention, and now sensors. I went through a few online tutorials what would be helpful in getting things going but many of them assumed some background in networking/setting things up, so I decided to write a tutorial from scratch in case someone else wanted to do this in the future. So, here we go (e-mail me if you have any comments/questions: You’ll need:

  • A raspberry pi3 or 0 (either one is fine, with pi3 you’ll be a little further ahead since it already has gpio pins, you’d have to solder them on the pi0 and buy pins that work for that),
  • 3 male jumper wires like these ones ,
  • 1 roll of lead for soldering like this one,
  • A soil moisture sensor. I bought mine at Sparkun, but there are others out there,
  • SD card with adaptor, and an adaptor in case you have the new Mac,
  • Electronic grade silicone for covering the wires, and
  • A basic soldering kit (the link here is over the top, you only need the soldering iron).
WhatsApp Image 2017-06-22 at 4.51.18 PM

All the materials you need to get a basic analogue soil moisture sensor running

Raspberry Pi Software: We’ll be The first is to make sure that your SD card has the right software for getting everything up and running. We’ll be using the full version of Raspbian (download of raspbian jessie with pixel). It’s a bit of a pain to do this in Mac or Windows (easiest is on Linux), but we’ll get there. Download the image, double click to unzip the file, and you’ll see the raspbian image that you need (for me it was ‘2017-04-10-raspbian-jessie.img’). Now we move on to the terminal:

Diegos-MacBook-Pro:~ diego$ diskutil list

This will give you a list of all your drives. You’ll probably see a “/dev/disk0 (internal)” and a “/dev/disk1 (internal, virtual)”.  Now, insert your sd card into the adaptor and insert that into the Mac sd card adaptor (annoying but necessary, it’s in the buy list above. Now, if you type “diskutil list” again you should see a new drive called “/dev/disk2 (external, physical)”. Check that the size of the drive is the same as the SD card. Now, lets unmount that drive (with [n] replacing the drive where the sd card was mounted (make sure that you copy and paste the correct name of the image you downloaded to avoid any mistakes!):

Diegos-MacBook-Pro:~ diego$ sudo diskutil unmountDisk /dev/disk[n]
Diegos-MacBook-Pro:~ diego$ sudo dd bs=1m if=~/Downloads/2017-04-10-raspbian-jessie.img of=/dev/rdisk[n]

It could take a little bit for the image to be burned into the SD card. When this is finished the terminal will stop doing work. Then just type:

Diegos-MacBook-Pro:~ diego$sudo diskutil eject /dev/disk[n]

Raspbian settings for wifi: Now that the image is burnt on to the SD card we will setup the wifi settings so you don’t have to setup an external monitor or keyboard to do work. One of the extremely annoying things about doing this on a Mac (and without a linux partition) is that when you try to access the burned image on the SD card you will only have access to one partition (the /boot partition) when you should have access to two. So, although there are a few things recommended to solve this problem on the internet many of them seemed a little bit hacky and painful, if you’re doing this as a one time thing go to Paragon Software and download their free trial. Install the software for your OS before proceeding to the next steps (if you proceed without having done something to be able to access your /ext2 /ext3 partitions you won’t get very far). Once the Paragon software is installed, remove and re-insert the SD card with the raspbian image and go to:

Diegos-MacBook-Pro:~ diego$ cd /Volumes

An if you “ls” inside that folder (ls allows you to see all the files inside that folder). For me, without Paragon (or any other strategy to see all the ext2/ext3 partitions) I was only able to see the boot folder, and a couple of other ones such as “Macintosh” “HD MobileBackups”, but after running the software a new folder appeared in cd /Volumes called “Untitled”. Now that we see that file inside the image we go into it and type the following things as follows:

Diegos-MacBook-Pro:~ diego$ cd Untitled
Diegos-MacBook-Pro:~ Untitled $ cd etc/
Diegos-MacBook-Pro:~ etc$ cd wpa_supplicant
Diegos-MacBook-Pro:~ wpa_supplicant $ sudo vim wpa_supplicant.conf

This gets you inside the file where we’ll add the wifi information (note that the ” ” are supposed to be there!). Note the absence of spaces next the =, make your user wifi and wifi password are typed correctly. The text that is in the wpa_supplicant.conf file could change a little depending on country and a few other things, but here is the example of what I had and wrote in mine (here are a few links that detail things that can go wrong when setting up your wifi network: link1, link2)



Now, before moving forward go to your /boot partition inside cd /Volumes and create a file inside of it called ssh (you create a file using the command “touch”), as follows:

Diegos-MacBook-Pro:~ diego$ cd /Volumes/boot
Diegos-MacBook-Pro:~ boot$ touch ssh

This last step is important so don’t forget to do it! Some mistakes that I did wrong when setting up the network included: 1)making sure that the password was correct, 2) making sure the  apostrophes I was using were correct (eg., ” vs ” , vim likes the ” straight ones), 3) wpa_supplicant file had the correct info, and 4) Making sure that the configurations on the network are accurate. You might have to edit both your wpa_supplicant.con and etc/network/interfaces files before succeeding, but this should give you a solid push forward. Note: 1) make sure that there are no spaces, and 2) make sure that you save your work in vim (:wq) before exiting.

If you succeeded at all these steps now you should be able to log into your raspberry via your wifi network! Now let’s try it. Remove your SD card and adaptor from your computer and plug the SD card into your raspberry pi (make sure the pi is turned on!). Wait for it to turn on, and then start looking for a connection to it. This is how you do it in your Mac terminal:

Diegos-MacBook-Pro:~ diego$ nmap -sP

This will print out a list of IP addresses where you might try to connect with:

Starting Nmap 7.50 ( ) at 2017-06-23 11:04 PDT
Nmap scan report for
Host is up (0.0093s latency).
Nmap scan report for
Host is up (0.016s latency).
Nmap scan report for
Host is up (0.0090s latency).
Nmap scan report for
Host is up (0.00087s latency).
Nmap scan report for
Host is up (0.084s latency).
Nmap scan report for
Host is up (0.057s latency).
Nmap scan report for
Host is up (0.0068s latency).
Nmap done: 256 IP addresses (7 hosts up) scanned in 8.54 seconds

You’ll have to try going one by one to each of these IP addresses until you find the pi (if you get a security warning just reply Yes, it will only ask you this the first time you’re connecting to it. The way you’ll look for the pi is by typing in the terminal (with [n] being the IP address that you’re trying to connect to):

Diegos-MacBook-Pro:~ diego$ ssh pi@[n]

In this particular case ssh pi@ is what worked for me, and then it asked me for a password (pwd: “raspberry” — without the apostrophes!). This should let you log in to the pi. If you’re successful it will be evident as your terminal user will change to (pi@raspberrypi:~ $). If you’re not successful in getting this, you need to check your wpa_supplicant.conf files (in the Linux partition, or however it is called in your computer) and the /boot partition. Every time you move the SD card from the raspberry pi to your computer the wpa_supplicant.conf file is changed by the pi so you’ll have to retype what we did above (make sure you save in the text file as work through things so it’s easy to copy and past) inside the wpa_supplicant.conf. The pi also deletes the “ssh” file inside /boot, so make sure that you go through the motions for wpa_supplicant.conf and touching ssh into boot before checking the wifi connections. This can be a little tedious and little mistakes can happen along the way so be patient!

Hardware: When you eventually manage to connect to the pi successfully, it is time to begin with the actual work. First, I would suggest creating git repository for your project and cloning into your pi so you can keep track of changes and you can easily track your work. As follows:

pi@raspberrypi:~ $ git clone

Then, install all the software and updates you’ll need to work on your project. Check here in case I forgot anything but this should have you covered for now:

sudo apt-get install vim #text editor that doesn't come in the pi
sudo apt-get update #important update for working with the pins
sudo apt-get upgrade #important update for working with the pins
sudo pip install twilio #important for sending SMS later on

After doing this we should be able to start playing with the pins! Before moving further lets take a look at the following sketch so that we wire things properly:


GPIO pins

For us, and looking at the labels in the soil moisture sensor will be connecting: 1) VCC –> 3v3 (Pin 1), 2) GND –> GND (Pin 9), 3) SIG –> GPIO 17 (Pin 11). So, before soldering anything together lets make sure that we are getting a signal (let’s pretend like in the picture below nothing is soldered together). Your cables are touching each of the holes of the soil moisture sensor and that you’ve connected each of their ends to the appropriate GPIO pins as I just indicated:

WhatsApp Image 2017-06-23 at 11.44.19 AM

I soldered this already, but imagine that the metal is just touching each of the holes and female portion of the wire is going into the appropriate GPIO pins.

Now, to see if we’re reading data lets go inside the pi’s python and look into what the pins are reading! Before going further I suggest you look into this quick introduction to python’s GPIO library as it will come in handy in the next few steps:

pi@raspberrypi:~ $ sudo python

and inside python lets import the libraries we’ll be using:

>> import RPi.GPIO as GPIO
>> from import Client

To test whether we are reading any data, grab a glass of water and dip the soil sensor and dip it in and dip it out. While you do that write the following code to establish what will be our input pins and whether or not we’re reading data (this particular soil moisture sensor is analog, so we’re only getting 0 and 1 values). Note that we’re making pin 11 our input pin:

>> GPIO.setmode(GPIO.BOARD)
>> GPIO.setup(11, GPIO.IN)
>> print(GPIO.input(11)) #When outside of the water this should print 0
>> print(GPIO.input(11)) #When inside of the water this should print 1

WhatsApp Image 2017-06-23 at 11.44.18 AM

Dip the moisture in and out of the water while printing the input fro gpio 11 as indicated above.

If this is not working make sure that all the libraries are installed and that you’ve connected the wires to the correct pins. It might take a little patience, but you’ll get there. Now, if you are successfully reading data lets go into the SMS part. Open an account with Twilio, buy a phone number, and make sure that in the Twilio settings you copy the “account_sid”, “auth_token”, and the phone number you just bought. Once you have this information it is really straightforward to write a script for checking soil moisture (1 or 0) and sending us a message when the soil is dry. Copy and paste the following script into a new python file (e.g., and save it in the git folder you create. Mine looks like this:

import RPi.GPIO as GPIO
from import Client
#Twilio setup
client = Client(account_sid,auth_token)

#GPIO Pin setup
GPIO.setup(11, GPIO.IN)

if GPIO.input(11) == 0:
message = client.messages.create(
body="Hello, it's the coconut :) please water me today!")

This should work and you should have received an sms saying “Hello, it’s the coconut :) please water me today!”. If you didn’t check your twilio settings and make sure that you copied all the settings correctly. Now that everything is ready, let’s solder the wires, and apply some silicon (or hot glue) on top of them so that they don’t move. The final step will be to create a crontab job so that this script runs every day checking when you’ll need to water the coconut. The final snippet of code for the crontab job is after this image:

WhatsApp Image 2017-06-23 at 11.44.20 AM

Soldering wires together.

To run this script on crontab you’ll  use the following code (note that I’ve let this be in UTC time, you’ll have to change the pi settings if you want to use your local time). Here is a short intro to crontab:

pi@raspberrypi:~ $ crontab -e #this will let you access crontab

Once you’re inside crontab just copy and paste this code (change the name of the .py if you used a different name). This crontab runs at 7.30 am every day and will only send me a message when I need water. Just edit crontab with vin and you’ll be good to go:

0 14 * * * cd soil_moisture_sensor; python

WhatsApp Image 2017-06-22 at 4.56.00 PM

A happy lettuce will grow from here at some point!


A Brief History of Oil in Mexico

In 1979, Mexican President Jose Lopez Portillo (1976 – 1982) announced to the world that Mexico had begun exploiting ‘Cantarell’, the world’s third largest oil field at the time (just behind the Ghawar and Burgan fields of Saudi Arabia and Kuwait). This bounty came with promises of jobs, technological development, commitment to industrialization, and sustainable city-building. Above all, Lopez Portillo (and his team of experts) stressed that this windfall of wealth would be reinvested in Mexico to guarantee a future ‘beyond Oil’.

It only took 24 years for Cantarell to reach ‘peak oil’ status. This means that in 2004 Mexico’s largest oil field had reached its maximum rate of petroleum extraction, after which it entered a state of terminal decline. The US and Norway ‘peaked’ in 1970 and 2001 respectively, and Brazil has become a net exporter of oil since 2009. Brazil’s recent success has been led by a stubborn decisiveness to dramatically reduce oil consumption, a strong emphasis on research and development, and a focus on non-fossil fuels alternatives for national consumption. Since 2003 it followed an aggressive ethanol blending policy – and today 80% of the cars in the country run on blended fuel.

It’s no surprise that Mexico’s ‘energy reform’ recently faced doubt and opposition. For decades, US, British and even Dutch companies exploited and extracted Oil from the country, in a time period marred by internal political conflict and tax evasion. It is also not clear how taxing PEMEX has benefited Mexicans – it is taxed close to 60% -, but it isn’t clear where the money has gone. It certainly hasn’t solved Mexico’s poverty, education, and ‘technical workforce’ woes. During the protests regarding the country’s most recent energy reform there was plenty of debate of how the money should be spent, but the attacks and questions seemed misled. Some important questions were not part of picture but are crucial for Mexico’s long term sustainability: Has oil done Mexico any good? Has the country suffered a resource curse? Has oil played a role in the country not moving or advancing forward? Where do we invest PEMEX funds? How do we take advantage of energy companies investing in Mexico? How can we ensure transparency in the sector and how do we track accountability? Will Mexico use the new found oil and gas resources for a future beyond oil? Or, will the country just let history repeat itself?

Paradox of Plenty: Innovation and Growth

The 2013 Global Innovation Index report describes different characteristics of countries that learn and innovate. There are 'innovation leaders' or those that have succeeded in creating well-linked innovation ecosystems where investments in human capital thrive in 'fertile and stable innovation structures'. There are also the 'innovation learners' - which include a group of 18 middle income countries that are 10% or higher above the trend line (Republic of Moldova, China, India, Uganda, Armenia, Viet Nam, Malaysia, Jordan, Mongolia, Mali, Kenya, Senegal, Hungary, Georgia, Montenegro, Costa Rica, Tajikistan,and Latvia) and demonstrate rising levels of innovation due to improvements in 'institutional frameworks, a skilled labour force with expanded tertiary education, better innovation infrastructures, a deeper integration with global credit investment and trade markets, and a sophisticated business community'. Countries like Uganda, Mali, Kenya, and Tajijistan have above-average performances, relative to their income class.

Costa Rica, Bulgaria, Montenegro, China, and Moldova, country's with relatively smaller income per capita scored higher innovation points than Mexico - how can this be? Similarly to other 'resource rich' countries in the report, countries endowed (or, historically endowed with) oil, gas, or some other natural resource crowd out investment in other productive sectors and hinder innovation. This phenomenon, known as 'the resource curse' or 'paradox of plenty' has been well documented historically. Some low- and middle-income oil rich countries usually use Norway as an example of 'what's possible when one is resource oil rich', but fail to realize that Norway is an extremely poor innovator, punching far below its weight relative to the amount of money that the country has.

It seems that being richer, and throwing more money at problems doesn't fix them. For an economy of over $1 trillion dollars, Mexico only has 15 venture-capital funds which only invested in 25 projects in 2011. The Bay Area alone (California) invested over $2 billion in the first quarter of 2013. What Mexican researchers and innovators need is support - institutions and incentives - that can fund projects from technology to market. Not empy promises.

Read More: The Global Innovation Index

Force-Directed Graph

Note: I grabbed the data from Mike Bostock’s wesite. I only used this example to embed my first d3.js graphic. The rest of the posts from now on, will be inspired by Mike, but will be product of my own work

This simple force-directed graph shows character co-occurence in Les Misérables. A physical simulation of charged particles and springs places related characters in closer proximity, while unrelated characters are farther apart. Layout algorithm inspired by Tim Dwyer and Thomas Jakobsen. Data based on character coappearence in Victor Hugo’s Les Misérables, compiled by Donald Knuth.