I said it first on Mastodon but I'll say it here again: I really do
not like X-ray.
X-Ray is a test management suite that integrates with the bug tracking
system you probably use whether or not you like it: Jira. And it is
damn near impossible to use correctly. I'm sure you use it too, or the
QAs at your workplace do. X-Ray looks great on paper. And if you're a
corporate exec with a bunch of check-boxes you need checked guess what
software will check them? X-Ray will.
But in practice it is frustrating and illogical to use. I have yet to
find a good workflow that will take me from an issue that needs tested
to test to approving that code for release to creating a test plan for
the release. I can get to the release test plan if I skip the logical
step of creating a test plan for the issue I'm testing, but if I
follow the logical flow of the issue I wind up performing a lot of
manual steps to get to the release. There is always some weird hard
stop or bit of logic that it doesn't handle. It makes me cranky.
And I get it. The developers were tasked with creating something that
integrated into Jira and managed tests the way Jira managed bugs. In
their position I'm sure I would have made a lot of the same decisions
they made. But for the love of god, it just doesn't work. When you
discover your premise is wrong you don't charge headlong into insanity
you change your course and do the right thing.
And tracking tests with your bug tracking software is the wrong thing
to do. Jira issues are ephemeral things. Once they leave the kanban
board they are forgotten until some poor product manager/data
archaeologist decides to unearth them again. And that's good. We
should be working to the future, not focusing on the past.
But my tests are tools, not issues. They need to be honed and
maintained. I can't do that when they are buried under thousands of
forgotten bugs. Or hundreds of layers of abstractions that are all
jira issues.
Each test scenario is a Jira issue. Those scenarios are organized
under test sets which are: Jira issues. When you want to run a series
of tests you create a test plan which is a jira issue. When you
actually run that test plan you create a test execution based on the
test plan. That execution is a jira issue. If you find problems and
have to run the test plan multiple times each of those runs creates a
test execution.
And of course these all need to link back to actual Jira issues so you
know which issue the issue that you organized under yet another issue,
attached to a fourth issue, and then took notes on in a fifth issue is
actually referencing.
X-Ray is maddening. And I don't think I'm going to use it anymore.
