I care about Ansible

I tell people that my first contribution to the [1]Ansible project was
[2]killing off the cows, but while researching for this post, I see
memory fails me: the first written record of me contributing to
Ansible is dated a bit earlier, on [3]July 20th, 2012, but that
doesn't matter. By the way, it was quite a bit later that [4]the cow
thing became configurable. Be that as it may, it's been almost seven
years - a long time and a good time.

I've contributed a number of modules to Ansible, quite a few lookup
plugins, documentation patches, and the piece de resistance was the
documentation (a.k.a. ansible-doc) system which is still used today. I
recall sitting on the floor in a conference center in Boston getting
ready to push what I lovingly called a "jumbo patch" which would
target each and every module: I had Michael's ok that he wouldn't
commit anything for 24h so that my patch would apply cleanly. It was
very exciting for me, having to wield a new utility (git) and hoping I
wouldn't mess anything up. A lot of adrenalin flowed. Good times.

Ansible's grown enormously since then, and I've not always been able
to pay attention to it as much as I'd have liked to, but I've used it
and have also had the pleasure of training a large number of people in
using Ansible. In the course of the last two or three years, I've met
two hundred people interested in how Ansible works. That is a lot.

During that time I've occasionally commented on a pull request,
responded to an issue, and I've even submitted at least one new lookup
plugin for consideration: it's the [5]lmdb lookup plugin.

I'm telling you this story to set a bit of background for what follows
and to demonstrate that Ansible is a project I've been very fond of.

If you look at that last link, you'll see I submitted it two years
ago, in May 2017. Since then, I've had to rebase the plugin once
because of changes which made it unmergeable. Judging by labels added
(by a human?) the plugin affects Ansible version 2.4 (current is
2.8.1), and it needs all sorts of things like a maintainer (huh?), a
revision, tests, and whatnot. Does that mean I should leave it as is?
Does that mean I should just close the PR and forget about it? I don't
know. I do know one thing: I will not rebase it again, and the powers
that be must just as well close the PR as I've completely lost
interest in it.

[6]tweet

Ansible is a hugely popular project; it's approaching 38K stars at the
time of this writing, and it has 16K forks. I don't know how many
people Red Hat has working for Ansible, but I guess there to be
several committers, and I know some of them by name and a couple
personally.

In my opinion they're being inundated.

At this time Ansible has 3900 open issues, and 1900 open pull
requests. The oldest open PR is dated March 2015, and the oldest open
issue is two months older. Never ever will it be possible to process
all that, if only because the code has changed so much meanwhile that
most of the PRs can likely no longer be applied cleanly, and I would
think a lot of the issues are no longer even valid. What I also expect
is that many (possibly first-time) contributors have meanwhile lost
interest in their contribution just as I've done for the lmdb lookup
plugin.

I was at a customer site recently a fortnight ago, and we had a devil
of a time because it appeared that the [7]archive module was omitting
files from an archive. Hard to believe, I know, but an empty file
gintonic in a directory-to-be archived, never appeared on the target
hosts. What?!

After a lot of testing, and upon beginning the tedious process of
submitting a bug report, I discovered [8]this bug was already reported
in January 2018. 1.5 years ago. This module error is not just some
cosmetic thing of a feature request. It is a bug which causes data
loss, and somebody added a waiting_on_contributor label to it, and I
am starting to wonder how long the wait is going to be, particularly
since a kind newcomer [9]provided a fix in February of this year. (It
also saddens me that this contributor may have been frightened off by
being requested to provide integration tests for the fix.) To be fair,
the archive module is a module maintained by the community which its
documentation clearly states; not being a core module, the Ansible
project proper is not responsible for it. So Ansible includes
batteries but some batteries are not be as powerful as others?

This is but one example which I use as it's the one which happened
most recently to me.

In the course of the past year I've spoken to a small handful of Red
Hat people about this. Unofficially I see a lot of nodding, and I see
some organization which has taken place in the project (repositories,
communities, etc.), but I don't see PRs and issues diminishing in
great numbers which could, of course, be due to an ever increasing
number of submissions.

I proposed a dramatic and drastic solution: close the 4000 issues,
close the 1900 PRs, and start over. Explain why this is being done.
(It'll be less painful to have a week of bad press than permanent bad
press.) Have people begin to raise issues and PRs again. Brutally
close whatever makes no sense. Merge the PRs quickly after a group of
peers have OK'd them. (e.g. 3 peers apply an OK, the PR gets merged,
somewhat how the OpenBSD project does it, I think). If an issue
remains unchanged for, say, 3 months, or a PR remains without activity
for that time, it's marked as stale and will be closed after a short
grace period. Ansible is an Open Source project backed by a strong
company. I for one would much prefer to be told "no, we don't want
that" than to be left hanging and guessing.

Ansible users are telling me their trust in Ansible is diminishing:
every release brings breakage (e.g. for Solaris, OpenBSD, a lot of
non-Linux). PRs are not being merged in a timely fashion, and issues
are not being solved. (Here's an example of an issue [10]addressed to
me, and I have clearly ignored it.)

This makes me very sad.

[11]Ansible Communities was an attempt by Dag Wieers to give people a
trusted process and some privileges to work independently on sets
of.modules, but it's not got a lot of traction, unfortunately.

The new [12]Ansible Collections, proposed in Ansible 2.8, could
alleviate the situation with modules by moving the modules to Galaxy
and to people's repositories. This would mean that Ansible in future
might just be a core with a minimal set of modules. My worry is that
the number of disparate versions of a particular module will explode:
contributors will go away, there'll be no central module repository
and that can mean dozens of versions of a particular module, each will
different features and/or patches. For example, searching Galaxy for
apache server filtered by Role produces 869 results, not all of which
really are an Apache server, and there are 10 roles to install an
[13]Mosquitto broker; that's a lot of choice... And many users I speak
to in Europe are quite uninterested in Galaxy; are you really willing
to pull configuration code you didn't write to blindly deploy onto
your servers? I [14]smell lots of cake ;-)

On the plus side, moving most of the modules out of Ansible will make
core developers happier, as they don't feel responsible for the dozens
of contributed modules anyway. All the more reason to clear out the
large number of issues and PRs.

Let's see what happens.

Update: Ansible responds to my blog post: [15]Thoughts on
restructuring the Ansible project.

References

  1. https://jpmens.net/2012/06/06/configuration-management-with-ansible/
  2. https://github.com/ansible/ansible/pull/1246
  3. https://github.com/ansible/ansible/pull/634
  4. https://github.com/ansible/ansible/pull/2755
  5. https://github.com/ansible/ansible/pull/24550
  6. https://twitter.com/kvlly/status/1137360516182151168
  7. https://docs.ansible.com/ansible/latest/modules/archive_module.html
  8. https://github.com/ansible/ansible/issues/34569
  9. https://github.com/ansible/ansible/pull/52117
 10. https://github.com/ansible/ansible/issues/22034
 11. https://github.com/ansible/community/wiki/Community
 12. https://galaxy.ansible.com/docs/contributing/creating_collections.html
 13. http://mqtt.org/
 14. https://jpmens.net/2013/02/12/sudo-bake-me-a-cake/
 15. https://www.ansible.com/blog/thoughts-on-restructuring-the-ansible-project

You are viewing proxied material from serf.jpmens.net. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.