Re: New packaging guidelines for Ruby

Vít Ondruch <vondruch@xxxxxxxxxx> · Fri, 02 Mar 2012 09:40:41 +0100

On Thu, Mar 01, 2012 at 10:05:58AM +0100, Vít Ondruch wrote:

Dne 29.2.2012 17:53, Toshio Kuratomi napsal(a):

    I've been thinking about this for a week and also waiting for people to
    bring forth packages that would be broken by it to see if there's any actual
    problems with the strategy.  No packages beyond the original one which is
    a bug from upstream has been presented.  Without that sort of example, I'm
    going to vote for #3.

Well it is hard to bring you some example, because there are non.

If there are no problems, then there's no reason not to use it.

There are no
experiences with the #3 except with rubygem-idn, and in this case the simple
approach described in guidelines fails. Try to remove the following lines from
the spec:

# Avoid "cert_chain must not be nil" error.
sed -i -e "10d" %{gem_name}.gemspec

and you'll see what I am talking about. Please do not ignore these custom lines
I had to come up with to let the #3 work.

I have not ignored this.  In the ticket, I stated that this looks like an
upstream bug which, as packagers, it's our job to detect and fix.
cert_chain is being set to nil by the upstream packaging.  This is not
supported by the current versions of the build tools (rubygems).  We update
so the build script to work with the current version of the tools (and
hopefully submit the change to upstream).

When gcc updates and code stops working, we patch the upstream code to work
with the new gcc.  When python updates, we patch upstream code of third
party modules so that they work.  There's nothing out of the ordinary with
this at all.

    #2 is more complex as people have to keep in mind two sets of instructions.
    When a bug fix is needed, it will not be simple to apply it as you'd have to
    change the whole structure of the spec file in order to apply the patch and
    then revert those changes when the patch is no longer needed.  The
    temptation for the packager in that situation will be to not apply bug fixes
    until after they've made it into an upstream gem release.

Yes, it will not be simple to apply changes, therefore we are moving from
approach #1 which was valid up until now and proposing the approach #2, which
works well for all packages except rubygem-idn

No.  You are proposing approach #2 plus approach #3.  So your proposal is
twice as complex as approach #3 as there's twice as many recipes for
building rubygems to learn.

    #2 does not fix the problem with rpmbuild --short-circuit.

To be honest, I do not understand what is the point with the --short-circuit.
According to documentation "--short-circuit skip straight to specified stage
(only for c,i)". I do not know how are you using it, but it implies that you
have to do "rpmbuild -bp" to be able to do "rpmbuild -bc --short-circuit". In
that case, what does not work for you? What do you want to see in this step if
it is empty?

rpmbuild -bp kernel.spec
cd kernel-3.0.0
# edit some source files, making backups w/ extension .myfix
cd ..
rpmbuild -bc --short-circuit kernel.spec
# If failure, edit source some more
# If success,: gendiff kernel-3.0.0 .myfix > myfix.patch and add it to spec
# file

    #3 was compared in the ticket with requiring "Fedora users to always rebuild
    each RPM locally, because there might be sometimes broken dependency or
    other error."  This is a wrong comparison.  A packager's job is to deal with
    broken dependencies and other errors, not a users.  #3 requires packagers to
    deal with these issues, not users.

You should understand that RubyGems is packaging system and if I am packaging
the gem for Fedora, I am user of the packaging system from point of view of
RubyGems and I am package maintainer form point of RPM. No need to argue about
"packager's job". I just wanted to find something what would give you better
feeling what is it about and I gave better, more detailed, example along the
lines somewhere in this thread to Rex.

Yes, you are the user of the upstream build system.  But it's not
upstream's build system that the end users of Fedora see.  So the comparison
that drags in end users is a wrong comparison.

Upstream build systems are sometimes easily adaptable to the way that
software should be packaged in rpms but at other times it is something that
we have to work around.  Take a look at packaging java code where upstream
has bundled multiple versions of jar files just for their package build as 
an example of something even more invasive than what we're talking about
here.

    I'm also seeing the argument made for #2 that it's less work because
    a single command exists to do all the steps.  Therefore we should use it.
    This is not a compelling argument because the same can be said of other
    build systems.  For most python code, for instance:

    %setup -n %{srcname}-%{version}
    python setup.py install --root %{buildroot}

    even "make install" is valid under this argument.

I am not familiar with python, but you cannot compare "make install" with "gem
install". It is something really different. Moreover, "make install" doesn't
necessarily mean that the "configure && make" will be run before.

The argument I'm seeing is that "Upstream provides me with a command to run
that upstream has told me to use to build and install the package.  It does
that in one step instead of spread over three steps like I want.  I could
do it a different way using the standard tools and then it will do things in
the proper three steps but I want to do what upstream has asked instead of
what is proper."  For that argument, the same can indeed be said about "make
install".

    #3 is also (barring someone giving us examples where packages are actually
    broken) only a small one-time cost.  Changing the rubygem spec files that
    are using the old guidelines to use the new guidelines.  As stated earlier,
    the FPC has always known that the Rubygem guidelines needed to be changed
    but was unaware that the time had come when the old guidelines were no
    longer needed (lutter hasn't been on the FPC for years).  Since you're
    proposing changes to the guidelines anyway, you can hardly complain about
    a one-time cost.  In terms of an ongoing cost, until you show that there's
    a non-bug reason that unpacking the source and rebuilding the gem is wrong,
    there's no extra cost in maintaining this that package maintainers shouldn't
    be responsible for anyhow.

Yes, you see just script which do conversion, but you do not count the
exceptions. Once more, I used #3 for rubygem-idn, the only gem which required
it and in this one case it immediately failed. So for me it is a *100% failure*
of your conversion script.

I've told you earlier why this is not an impressive statistic.  Even if
*all* rubygem packages were to fail because they're using features of
rubygems that have been removed in the current version of rubygems, it would
be beneficial that we had detected the problem and submitted patches to the
builds to upstream.  So in a way the fact that this bug in rubygem-idn would
have passed unnoticed if you didn't have to fix something else is a failure
in your proposed guidelines as it's not being detected until unrelated
changes are made.

From last versions of packaging guidelines we moved from #1 to #2 which solves
practical problems. Move from #2 to #3 solves some artificial problems.

That is incorrect.  #2 is incomplete by itself.  You must include #3 in
order to have a workable guideline.  OTOH, #3 is complete in and of itself.
So leaving off #2 halves the number of specfile variations they need to know
if they want to package only rubygems.

Yes, I agree with you that if differs a bit from other packages. Yes, I agree
with you that it could be better. Yes, I proposed that I will work with
upstream to allow better way of patching C extensions, without need to move to
#3. Neither of this seems to satisfy you.

That is correct.  A workflow that looks like #3 is the correct way to
package software.  You are not proposing to work with upstream to make that
possible but instead working with upstream to kludge their current broken
process to do one more thing (patching C extensions) in a broken manner.

The proposed #3 is the workflow that the package build should have.  If
you're unsatisfied with that, you should be talking to upstream about how to
adapt their build system to meet that workflow in all cases.

Please note that this [1] was the last proposal coming from me and Ruby-SIG. I
am not against formal changes of the guidelines, as was done by Toshio, if FPC
believes they will be better aligned with other guidelines. I never said
nothing against. I totally support that effort. When I was asked to come up
with the way how to patch the C extensions and I did that. However I am
strongly against mandatory #3.

But you have given no examples that show why it would be wrong.  And without
that, you're just spreading fear that a mandatory #3 will cause you issues.

If you're so sure that you can get close to a 100% failure rate out of #3
that is due to problems with the procedure rather than bugs in upstream
packaging, you should be able to pick a random rubygem, adapt it to #3 and
then expose some new type of bug that we can examine and determine why it's
unworkable.  Failing to do that, the argument that it won't work is just
speculation, not fact.

-Toshio
--
packaging mailing list
packaging@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/packaging
--
packaging mailing list
packaging@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/packaging