~sturm

Ballarat, Australia

https://stumbles.id.au/

Free software activist, software engineer and director of Sturm Software Engineering. Loves riding bikes and flying kites. Maintainer of GNU MediaGoblin.

#25 Replace py-bcrypt dependency with bcrypt 10 days ago

Comment by ~sturm on ~mediagoblin/mediagoblin

Fixed by ~eroca. Thank you!

REPORTED RESOLVED FIXED

#23 Provide an easier way to get the permissions right in deployment 10 days ago

Comment by ~sturm on ~mediagoblin/mediagoblin

I've updated the commands at the bottom of the "Nginx as a reverse proxy section" to include the above chmods, plus one for fixing the virtualenv bin directory.

#26 Add link in deployment docs to --system-site-packages rationale 10 days ago

Ticket created by ~sturm on ~mediagoblin/mediagoblin

Using a virtualenv is a bit of a tricky compromise, because it means that while we benefit from security updates in the OS-packaged versions, updates can also break compatibility with the pip installed packages. I'd like to make sure we really spell out this rationale in detail compared to say not using --system-site-packages or using all OS-installed packages on compatible operating systems.

#25 Replace py-bcrypt dependency with bcrypt 13 days ago

Ticket created by ~sturm on ~mediagoblin/mediagoblin

The py-bcrypt package is unmaintained and has been removed from Guix.

#14 Does pyconfigure even make sense anymore? 22 days ago

Comment by ~sturm on ~mediagoblin/mediagoblin

I asked for an opinion on #pypa IRC about packaging applications:

  • sturm I'm working on packaging for the MediaGoblin project - which is a Python web app. I think it makes sense to distribute a PyPI package, but we have both translations to compile and some third-party JS libs to bundle. Conceptually, would we we pre-build the translations and JS bundle, then include them as data files for subsequent python -m build? There's no way to add extra steps in python -m build right? It's
  • sturm just for preparing the final distribution?
  • webknjaz You need to learn about PEP517 in-tree build backends — this will allow you to add some code when producing artifacts.
  • sturm webknjaz: great, thanks I'll go do some reading
  • webknjaz OTOH it's questionable that you really need to distribute your app. Let me check what that is
  • sturm webknjaz: I'd be very interested in your thoughts on whether a PyPI package is even a good idea. It seems like a grey area for applications.
  • webknjaz Yep. I think it heavily depends on the application and the intended use of it.
  • sturm webknjaz: any rules of thumb you'd suggest in deciding?
  • webknjaz Well, libs and frameworks go to PyPI, and they have open-ended direct (usually only direct) deps specified.
  • webknjaz For apps, it's different. If you expose some sort of a CLI (or runpy) executable, you could probably use PyPI as well.
  • webknjaz But normally apps need stricter env specification.
  • webknjaz Like exact pins of the whole dependency tree (including transitive deps).
  • sturm stricter? as in only targeting a particular platform?
  • sturm ah I see
  • sturm we've been trying to keep pins very loose to hopefully ease distro packaging
  • webknjaz Normally, this is a task for virtualenvs and requirements+constraints files, and the assumption that the users probably wouldn't attempt to install incompatible deps/packages into the said venv
  • sturm that's a good point though, your existing PyPI release could break at any moment if a new incompatible dependency was released and you hadn't fully pinned it down.
  • webknjaz That's why you mostly need to package a virtualenv (or something similar).
  • webknjaz And there's a lot of tools that solve this problem in different (although similar) ways
  • webknjaz There's zipapps (natively supported by the Python) interpreter
  • webknjaz And tools like shiv or pex for creating them
  • webknjaz Also, there's things like py2exe and similar, that produce OS-specific bundles even including a separate interpreter copy sometimes
  • sturm ok, thanks - I'll look into that
  • webknjaz FYI we maintain this documentation website https://packaging.python.org/
  • webknjaz You should be able to find more pointers there
  • sturm is the main hassle you'd expect for distributing an app on PyPI the problem of getting the many deps to align/remain compatible?
  • sturm or are there other issues too?
  • webknjaz Also, here's a few more specific pointers https://packaging.python.org/overview/#packaging-python-applications https://packaging.python.org/discussions/deploying-python-applications/
  • sturm great, thanks
  • webknjaz As for your last question: when you publish an open-ended thing, be prepared to continuously maintain/test it as the dependency tree will be changing over time (each new user install will attempt to download the latest deps, not those you tested against at the time of the publication)
  • webknjaz Also, you'll be shifting the responsibility of managing the transitive dep tree pins in the installation locations to the end users.
  • webknjaz I guess, if the end-users are pythonistas, it's not very problematic, just point them to pip-tools as an example solution. But you may end up having to educate folks who don't understand that they'd need to manage a proper venv. OTOH now that we've adopted pipx, it may ease some of the issues with attempting to install incompatible things.
  • sturm right, makes sense. Is there any particular reason that dropping all the transitive deps into setup.py/cfg would be a bad idea?
  • sturm (aside from that it's not generally done)
  • webknjaz Well, normally, the right place for this is requirements. And pip-tools is not able to generate them in setup.cfg/py
  • webknjaz But it may be fine if you tell the users to install with pipx
  • sturm webknjaz: thanks very much, that's all really helpful - appreciate your time :)
  • webknjaz Another trade-off with the pinned deps in the libs is that it would be hard to get security fixes somewhere deep in the tree. Every time you go this route, you will need to have a mechanism in place to make new releases.
  • webknjaz You're welcome :)

#24 Provide an easier way to get the permissions right in deployment 27 days ago

on ~mediagoblin/mediagoblin

REPORTED RESOLVED DUPLICATE

#24 Provide an easier way to get the permissions right in deployment 27 days ago

Ticket created by ~sturm on ~mediagoblin/mediagoblin

For example:

find /opt/mediagoblin/user_dev/crypto -type d -exec chmod 750 {} \;
find /opt/mediagoblin/user_dev/crypto -type f -exec chmod 640 {} \;
find /opt/mediagoblin/user_dev/crypto -type d -exec chmod 750 {} \;
find /opt/mediagoblin/user_dev/crypto -type f -exec chmod 640 {} \;

Plus also need chown. Can we spell out the chmod commands to avoid the numbers? These are totally opaque to new users eg. "grwx"?

#23 Provide an easier way to get the permissions right in deployment 27 days ago

Ticket created by ~sturm on ~mediagoblin/mediagoblin

For example:

find /opt/mediagoblin/user_dev/crypto -type d -exec chmod 750 {} \;
find /opt/mediagoblin/user_dev/crypto -type f -exec chmod 640 {} \;
find /opt/mediagoblin/user_dev/crypto -type d -exec chmod 750 {} \;
find /opt/mediagoblin/user_dev/crypto -type f -exec chmod 640 {} \;

Plus also need chown. Can we spell out the chmod commands to avoid the numbers? These are totally opaque to new users eg. "grwx"?

#22 JPEG thumbnails have visible compression artifacts 27 days ago

Ticket created by ~sturm on ~mediagoblin/mediagoblin

I've noticed that the JPEG thumbnails on many sites have visible artifacts that could be improved by a higher JPEG quality. We may also want to consider offering a higher resolution version for higher pixel density devices.

Here's some examples:

#18 Ubuntu CI build 28 days ago

Comment by ~sturm on ~mediagoblin/mediagoblin

Done!

REPORTED RESOLVED FIXED