TL;DR Stone is available on PyPi

pip install stone-site

Getting on that PyPi in the Sky

After writing the last blog post I got a little irritated at having to use stone in the following way:

cd ~/projects/stone
python3 stone ../half_systems/

This wasn't and isn't an issue but it bothered me enough that I decided to clean stone up. I took to the interwebs and searched for standard Python package layouts and how distutils/setuptools worked. A late night and some name conflicts later stone was available on the Python Package Index (PyPi).

Easy as setup.py

So what is setup.py and how do you use it? setup.py is one of the more important files in your project and it should live in the root of your project. Both distutils and setuptools use it to configure your project for installation. Here is the obligatory StackOverflow link to explain the difference between distutils and setuptools.

So how do you get started with setuptools? The Python Package Authority (PyPa) have made that fairly straight forward. They created this sample project to show developers, how to get started.

What is in setup.py anyway?

The minimal setup for setup.py is:

from setuptools import setup, find_packages

setup(
    name="HelloWorld",
    version="0.1",
    packages=find_packages(),
)

This is the least you need to use setuptools. The more data you provide to setuptools, the more is available on PyPi and to your users. There are additional options for including/excluding project files and build options for mixed language packages.

So after setting up setup.py for stone, I could finally call it like so python3 -m stone <dir>. Oh. Oh, dear, that is what I wanted but at least I don't have to be in the project directory anymore.

After I had a quick google, I found a few posts describing how I should set things out make stone work as I wanted. Python Apps the Right Way: entry points and scripts by Chris Warrick and this StackOverflow question.

I decided to also search for a few examples of good packages and follow their lead. I modified stone to conform with howdoi and flask's layouts.

.
├── CHANGES.txt
├── LICENSE
├── README.md
├── requirements.txt
├── setup.cfg
├── setup.py
└── stone
    ├── __init__.py
    ├── __main__.py
    └── stone.py

Once I had completed this, I set up the entry_points section in setup.py as follows:

entry_points={
    'console_scripts': ['stone=stone.__main__:main']
}

Now stone is callable on the command line as stone <dir>.

Distributions? Pure Bloods? Wheels?

I’m not going to cover all the differences between source distributions, universal wheels and platform wheels. I have a bit understanding of them but you should read the documentation.

If your project is pure python you should be able to get away with:

python setup.py sdist bdist_wheel --universal

sdist will generate an archive of just your source code. bdist_wheel --universal for 'Pure Python’ packages will compile the project. This compiled project creates a wheel which is faster to install for end users.

To Twine or not to twine

To upload a package to PyPi you will need to create an account.

This is where things got a bit confusing.  The Python community needs to clean up the documentation and tools in this area - They might be already but  ¯\_(ツ)_/¯.

You will need to create a ~/.pypirc file, This files stores your username and optionally your password. If your password is not stored you will be prompted for it later. Storing your password in plaintext isn't the best idea. However, if someone did have access to your home folder, you probably have bigger issues to worry about.

[pypi]
username = <username>
password = <password>

After having configured your setup.py you're discouraged from using it to register and upload your package.

python setup.py register  
python setup.py upload

Instead, you should use twine to upload your package, twine will automatically register your package if the name is available:

pip install twine
twine upload dist/*

That having been said and done. You probably want to test all of this out before you upload straight to PyPi thankfully there is testpypi for testing the process out. testpypi is cleaned semi-regularly, so you shouldn't worry about making a mess.

Before we can use it we need to update our ~/.pypirc to handle multiple index servers:

[distutils]
index-servers=
    pypi
    testpypi

[testpypi]
repository = https://testpypi.python.org/pypi
username = <your user name goes here>
password = <your test password goes here>

[pypi]
username = <your user name goes here>
password = <your live password optionally goes here>

Unfortunately, testpypi uses an older interface. It doesn't support twine's auto registration. So you need to use the unrecommended method of registering to test your package.

python setup.py register -r https://testpypi.python.org/pypi
twine upload dist/* -r testpypi
pip install -i https://testpypi.python.org/pypi <package name>

Congratulations you can now publish on PyPi!

One More Thing

Stone now has a GitHub project board