Raspberry Pi Python Adventures

Tip #8

Always use a good bit of data to test your data driven apps. Don't rely only on nose testing. But where to get data? Fake it. Never underestimate the power of import random. But when you need more than numbers:

pip install fake-factory

You can also take a look at faker, faker.py, ForgeryPy (and many more on pypi.python.org). Then there is fake-data-generator. Or if you want a csv or sql, try mockaroo.com

What it does: Although you could use real data, sometimes you don't have any. In fact, more than likely you probably wont be  able to generate a significant amount of data for weeks after going live with your web application. Or perhaps it is a desktop application and you'll never see the generated data. So just fake it. You need volume, and it's easy to create.

Another point to keep in mind is that using real data might be risky, depending on what it is. For sure you do not want real credit card numbers floating around on development instances.




François
@f_dion

Tip #7

Today's tip is quite basic, but will require time and effort to master:

Master the shell environment

What it does: Mac, Windows, Linux, BSD or Unix (or even something else). Whatever your operating system, become really good at using the command line, the shell. Bash, Powershell, ksh93 etc. Learn it. Else, it's like learning a bunch of words in a new language, but never learning the correct constructs. You might be able to communicate, but it'll never be very efficient. So go and find tutorials.

And then find the tools that'll make your life easier.

For example, *nix users, are you familiar with autojump (plus it's written in python)?

Windows users, did you know there is an equivalent Jump-Location for powershell?


François
@f_dion

Tip #6

Today's tip is in response to a great question on a local Linux user group:

python -m cProfile myscript.py

What it does: It'll give you a breakdown per line of how much time each operation takes to execute. Normally, profiling is best done with something like dtrace, to minimize the impact on the run time, but the original question was about figuring out the time for each operation in a python script running on the Raspberry Pi (no dtrace...).

Assuming the following script (we'll use sleep to simulate different runtime, and not call the same function either, else each would be collased under one line on the report):

from time import sleep

def x():
    sleep(4)

def y():
    sleep(5)

def z():
    sleep(2)

x()
y()
z()
print("outta here")

we get:
python -m cProfile script.py
outta here
         8 function calls in 11.009 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   11.009   11.009 t.py:1(<module>)
        1    0.000    0.000    4.002    4.002 t.py:3(x)
        1    0.000    0.000    5.005    5.005 t.py:6(y)
        1    0.000    0.000    2.002    2.002 t.py:9(z)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        3   11.009    3.670   11.009    3.670 {time.sleep}


François
@f_dion

Tip #5

Meet the triumvirate  of python interactive sessions:

help(), dir(), see()

So no doubt you use help, and probably dir, but you are probably wondering about see()... That's because it has to be installed first:

pip install see

What it does: Unless you speak native dunder (double underscore), dir's output can be a little overwhelming. For example, a dir on an int object (everything is an object in python...) gives us:

>>> dir(1)
['__abs__', '__add__', '__and__', '__class__', '__cmp__', '__coerce__', '__delattr__', '__div__', '__divmod__', '__doc__', '__float__', '__floordiv__', '__format__', '__getattribute__', '__getnewargs__', '__hash__', '__hex__', '__index__', '__init__', '__int__', '__invert__', '__long__', '__lshift__', '__mod__', '__mul__', '__neg__', '__new__', '__nonzero__', '__oct__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdiv__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'conjugate', 'denominator', 'imag', 'numerator', 'real']

>>> from see import see
>>> see(1)
    +           -           *           /           //          %           **
    <<          >>          &           ^           |           +obj
    -obj        ~           <           <=          ==          !=          >
    >=          abs()       bool()      divmod()    float()     hash()
    help()      hex()       int()       long()      oct()       repr()
    str()       .conjugate()            .denominator            .imag
    .numerator  .real

A little more human readable, no? Oh, I'm about to hear the complaint about typing from see import see everytime you start up python. Time to go and check tip #2...


François
@f_dion

Tip #4

I was mentioning cppcheck on twitter, for those of us who also code in C/C++. I must admit I didn't start using it until I saw Alan (Coopersmith) using it on Xorg about a year ago. So, what do we have for python? Today I'll make a quick mention of Pylint. Install is simple, along the line of (adjust to your package manager):

sudo apt-get install pylint

Then you can go into a python project and do:

pylint your_filename.py

What it does: "Pylint is a tool that checks for errors in Python code, tries to enforce a coding standard and looks for bad code smells", according to pylint.org. It also gives your code an overall mark. It's a good idea to at least run it and look at the suggestions it offers.

Bonus: pylint includes pyreverse which allows one to generate a package and class diagram (UML) from source code. This works ok as long as the code is straight forward.


François
@f_dion

Tip #3

As you install new modules (say, with pip install) to support your Python application, add them to a requirements.txt file and do the same for your tests, as test_requirements.txt. Installation is then a simple:

pip install -r requirements.txt

What it does: It allows you to keep track of what packages are needed if you share your code, deploy it to other machines, or if you somehow have to rebuild your computer. You can also quickly test that the list is up to date by creating a virtualenv --no-site-packages, and then using a pip for that virtual environment to do the install.


François
@f_dion

Tip #2

In your home directory, in a file named .env.py put the imports you want to always have preloaded in python interactive mode:

from some_module import something

In your .bashrc or .profile, add:

export PYTHONSTARTUP=$HOME/.env.py

What it does: When you login and open a terminal, the environment variable PYTHONSTARTUP will be set, and when you execute python (or bpython, too), the python interpreter will load whatever scripts are in PYTHONSTARTUP and be ready for you to use them without having to type them everytime. In this example, I could use functionality something of some_module right away.


François
@f_dion

PTOTD

Starting tomorrow, I'll post a Python tip on a regular basis. I cant promise a PTOTD, but it'll be more often than once a month, so that identifies the boundaries.

Ok, I lied, I'll start with one right now:

Tip #1

python -i script.py

What it does: At the conclusion of the execution of script.py, instead of exiting, the python interpreter stays in interactive mode, with everything ready to be printed or debugged.

François
@f_dion