fidzu : Gentoo

Recently with 1.7.9 release git gets support for git commit signing. This was one of the last show stoppers to migrate gentoo-x86 tree from cvs to git. In recent portage versions like 2.1.10.45 and 2.2.0_alpha85 support for git gpg signed commits was also added. So in kde overlay we decided to test with feature with thin manifests.

New repoman can use git gpg signed commits if you do the following thing

1. Enable special tag in metadata/layout.conf
   sign-commits = true
2. Install git >=1.7.9
3. Configure your gpg sign key for git via
   git config --global user.signingkey $KEYID

After first commit you can chek that gpg signature was added to special field in commit via

git log --show-signature

It will look like this

commit 9b3cafc7efb2c17b0f2baffae530196014967921
gpg: enabled debug flags: memstat
gpg: Signature made Sun Feb 5 21:06:06 2012 MSK using RSA key ID F82F92E6
gpg: Good signature from "Alexey Shvetsov "
gpg: aka "Alexey 'alexxy' Shvetsov "
gpg: aka "Alexey 'alexxy' Shvetsov "
gpg: aka "Alexey Shvetsov "
random usage: poolsize=600 mixed=0 polls=0/0 added=0/0
outmix=0 getlvl1=0/0 getlvl2=0/0
secmem usage: 0/32768 bytes in 0 blocks
Author: Alexey Shvetsov
Date: Sun Feb 5 21:05:48 2012 +0400

[metadata] Enable git signed commits by default

Signed-off-by: Alexey Shvetsov

05 Feb 2012 1:29pm GMT

Gentoo Prefix is still alive and going strong. In my opinion, Gentoo Prefix remains a strong point of Gentoo Linux and really establishes that Gentoo Linux is a metadistribution. In this post I want to focus on the numbers. The number of packages in the Gentoo Prefix tree, specifically. But first, a history lesson. It wasn't until EAPI3 in Gentoo that "allowed" Gentoo Prefix variables into the main Gentoo Linux tree. That was in late 2011, but Gentoo Prefix existed much before then, all the way back to 2006 (at least). Before EAPI3, the prefix team made slight modifications to ebuilds and placed them in a repo and called it the tree of packages for Gentoo Prefix. This worked fine, but we had growing pains. The major issue was that we were getting too successful to manage the increased contributions from users. In other words, as the number of "forked" packages grew, the amount of maintenance time increased greatly - this is due to the fact that it is a chore to keep our forks synced. At least, a large chore for a small team. This is why we looked for help and adoption from the other pool of 200 Gentoo Developers, hence EAPI3 and beyond. Since supporting Gentoo Prefix is not a big use of overall developer time, this has gone over quite well in my opinion - yes, there are some pain points at times I do realize. Enough history, here are the numbers:

• Number of packages in Gentoo Linux: 15554 packages in 154 categories.
• Number of total* packages in Gentoo Prefix: 9483 packages in 154 categories.
• Number of KEYWORDED packages in Gentoo Prefix: About 3000 for the most popular arch
• Number of packages still NOT in the main Gentoo Linux tree: 369 packages

* The total packages in the tree also contains non-keyworded packages because that just makes life simple. Once packages started migrating to the main tree, I helped think of this "whitelist" concept. The short version of the whitelist is that if a package is listed in that text file, it gets included in the Gentoo Prefix tree as a direct copy of the version in the Gentoo Linux tree. The presense of the package in the old repo means that it is used instead. Eventually, this concept will go away and we will overlay the Gentoo Linux tree directly.

So why is it taking so long to migrate ALL packages to the Gentoo Linux tree? Well, that is where the rubber meets the road and we get into roadblocks. A roadblock for us could be a number of things, such as a disagreement with the Gentoo Linux maintainer, some patches existing that we don't feel are a good fit for Gentoo Linux, or even us being lazy and not submitting stuff to upstream. We also don't want to push invasive changes to Gentoo Linux for critical packages, like the toolchain for example.

It has long since been our agenda to not add anymore packages to the old repo and going forward only adding new stuff to Gentoo Linux directly. I hope we can make a dent in those remaining 369 in 2012!

03 Feb 2012 4:48pm GMT

If you do, maybe you want to consider proxy-maintaining it as it now on its way out. Upstream has a much newer version available, and we in the Proxy Maintainers team will be glad to steer you in the right direction when you need the help.

Just send us an email.

02 Feb 2012 10:52pm GMT

I keep getting e-mails asking me why app-text/xpdf is masked for removal from the portage tree. It's getting too much to reply individually, so let me sum up the situation here in a blog post.

# Andreas K. Hüttel <dilfridge@gentoo.org> (27 Jan 2012)
# Has developed into an unmaintainable mess, and everyone who
# knows about it is either retired or missing in action.
# Several minor bugs and one ugly security issues (#386271).
# Masked for removal because of lack of maintainer.
# Please try app-text/epdfview as light-weight replacement.
app-text/xpdf

Xpdf is a package with a long history, and in a way a strange remnant of bygone times. Since PDF rendering is a function that many different programs could use, some years ago the Poppler library was forked from the Xpdf codebase. By now, Poppler is a much more active project, and used by dozens of packages in the Gentoo portage tree, all the way from LibreOffice and PDFTeX to Calligra, GIMP, and e.g. Okular or Evince. Being the more active project is important in this case, because PDF files are frequently shared and distributed and PDF rendering is thus a security-relevant task.
The original Xpdf remained independent of Poppler, not using the library - with the effect that every now and then security bugs kept popping up. Some time ago, some Gentoo developers started modifying and patching Xpdf to use the Poppler library. What resulted was the complicated construct that right now noone here is willing to maintain anymore. (Otherwise some Gentoo developer would have contacted me in the meantime.) Implementing a version bump to a more recent Xpdf version is a non-trivial task because all the Gentoo-specific patches have to be reviewed and if necessary rewritten.
Thus, app-text/xpdf needs to go the way of the dinosaur. Two alternatives exist, but both do not seem realistic at the moment:
1) We could go back to the original, unpatched Xpdf from upstream. I'm not going to do it, and I doubt anyone else of the Gentoo devs will.
2) Rogério Brito has started maintaining a fork of Xpdf at Github, which uses the Poppler library. However, there is no released version yet, and as he told me myself, he's rather busy in real life right now...
In the meantime, please try one of the following packages:

• app-text/zathura (based on poppler & gtk+)
• app-text/apvlv (based on poppler & gtk+)
• app-text/epdfview (based on poppler & gtk+, ~unmaintained)
• app-text/mupdf
• app-text/gv (based on ghostscript)
• app-text/evince (based on poppler, Gnome application)
• kde-base/okular (based on poppler, KDE application)
• app-text/acroread (yes I know...)

Ironically, the first mail reply to the last-riting of xpdf was from one of our security team members, promising me a beer the next time we meet in person. Only afterwards the complaints started.

02 Feb 2012 10:33pm GMT

sed -i 's/gregkh@suse.de/gregkh@linuxfoundation.org/g' .addressbook

01 Feb 2012 5:08am GMT

As discussed in the last Gentoo Qt meeting, we moved our overlay from gitorious to git.overlays.gentoo.org. This is going to be the final move, I promise

Along with that, we decided to change the overlay from qting-edge to just qt. Layman list is alreay updated, so if you still have the old one, you should remove it and add the new one:

# layman -f
# layman -d qting-edge
# layman -a qt

Keep in mind that this overlay contains mostly live ebuilds of Qt (branches 4.7 and master), so make sure that you really need it before blindly adding it (the same applies for the kde overlay). Enjoy!

31 Jan 2012 10:24pm GMT

1. Roll call

johu, hwoarang, pesa, tampakrap, wired

2. Qt 4.8

* cairo fails to build, patched ebuild available in qting-edge, #380013

Cairo build issue is fixed in qting-edge overlay, will be moved together with Qt 4.8.0 to tree.

* qt now defaults to the raster graphicssystem, we should remove raster USE flag, #398283

Wired created a eselect module to choose the Qt graphicsystem. Raster is default, other selectable are opengl, openvg and native. Raster use flag is not needed anymore, qt-gui depends on the new eselect module.

* do we really want to keep qpa USE flag?

qpa and c++0x will be masked in tree.

* are we going to fix #363939 for 4.8?

Wired fixed this bug in qt 4.8.0. Qt 4.8 will be moved to tree on next weekend. Dilfridge prepares kde-base/kstyles-4.7.4 to be rebuild together with Qt 4.8.0 to prevent crashes in KDE apps with Oxygen style.

3. Minor arches and Qt >= 4.7

Upstream supports official amd64, arm and x86, but other arches also considered in configure script. Keep stable keywords for minor arches in Qt 4.6. Wait for minor arches arm, ppc, ppc64 in current stabilization in Qt 4.7.4. Drop sparc keywords in Qt 4.8.0.

4. Overlay migration to git.overlays.gentoo.org

Tampakrap will set up overlay on git.overlays.gentoo.org on next weekend. The new overlay will be renamed to qt instead of qting-edge.

5. Open bugs

* #398885 qdoc3 broken on arm

We will ask the reporter if it works when he builds manually by providing him a configure command to make sure he tries the proper build.

* #394533 Libreoffice crashes in qt on exit

Can't be reproduced with Libreoffice 3.5.0.1, seems to be resolved by upstream.

* #392433 desktop file name issues

Will be fixed in Qt 4.8.0, so that qt-gui and qt-assistant no longer pass absolute paths to make_desktop_entry().

* #388551 qt-gui[gtkstyle] should depend on gnome-base/libgnomeui-2

We will add a elog message in qt-gui[gtkstyle] saying that for things to work you either need libgnomeui or that variable set properly in your env.

* #382559 qt_mkspecs_dir() returns bad spec directory

The bug will be marked as RESOLVED WORKSFORME, because we can't reproduce it. Additionally we change the eclass not to use LIBDIR in favor of get_libdir() after Qt 4.8 hits the portage tree.

* #359391 qt4-build.eclass should check for -buildpkgonly before downgrade sanity check

Resolution will be RESOLVED WONTFIX. Sanity check is there for a reason. It's not a matter of source or binary downgrade.

31 Jan 2012 7:18pm GMT

Since one hour ago, Qt-4.8.0 is in Gentoo portage tree. New major release so lots of new (or broken) stuff. The most important feature in this release is the integration of a new eselect module. This module will allow you to set your default graphics engine without the need to recompile Qt (x11-libs/qt-gui to be precise) from scratch. So, provided you have qt-gui-4.8.0 installed, you should be able to use the eselect module as follows:

hwoarang@mystical ~$ eselect qtgraphicssystem list

Available Qt Graphics Systems:
 [1] native
 [2] opengl
 [3] raster *

(note: if you have x11-libs/qt-openvg installed, one more option should be available)

Simply select your graphics system of preference, and then logout and login again.

hwoarang@mystical ~$ eselect qtgraphicssystem set 2
Setting opengl as your active Qt Graphics System... done
Please logout for changes to take effect.

Thanks to Alex(wired) for the eselect module implementation.
Enjoy ;-)

29 Jan 2012 7:46pm GMT

A small notification to tell you that the SELinux policies that were pushed to the main tree 30 days (or more) ago have now been stabilized (none of them introduced problems, although some of them have other bugs still open which are either fixed in ~arch or will be fixed in the hardened-dev overlay soon). I'll be working on pushing an additional set of changes to hardened-dev overlay today as it includes fixes for openrc that are quite important, and might even push this to the tree faster than usual.

The reference policy is also working on a new release, so the moment it is released we will be picking that up as well (give or take a month, since my availability will be a bit less the next month).

29 Jan 2012 11:33am GMT

In this podcast, create a best off cd with soundconverter and gnomebaker. The new Gentoo LiveDVD with persistance. The Northeast Linux Fest Saturday March 17, 2012, Worcester MA. Samsung ML3312 and Linux plus an Interview with Milan Kazarka.

Links

Northeast Linux Fest | Saturday March 17, 2012.
http://www.northeastlinuxfest.org/

Gnome Shell Extensions
https://extensions.gnome.org/

Gentoo 12.0 LiveDVD
http://www.gentoo.org/news/20120102-livedvd.xml

Samsung ML-3312ND
http://www.samsung.com/us/support/downloads/ML-3312ND/XAA
http://gpo.zugaina.org/net-print/samsung-unified-linux-driver

Interview with Milan Kazarka
http://www.gentoo.org/news/20120119-milan-interview-announcement.xml

Download

28 Jan 2012 10:46pm GMT

You probably remember my previous notes about Wordpress, FTP and the problem with security. At the end after a (boring) set up session I was able to get vsftpd provide FTPS service, which should be usable both by Wordpress and by Dreamweaver, so that my friend the webmaster can upload through it directly.

This is important because as it happens I have another prospective customer who's going to run Wordpress, and FTPS now start to look more interesting than SSH, as it doesn't require me to give shell access to the server either.

Unfortunately I'm a bit worried (maybe more than I should be) for the use of standard passwords rather than certificates or keypairs for authentication. Which meant I went tried to think of other alternatives.. of which there are mostly two: Google Authenticator and YubiKey .

The latter I knew by name already because I proxy-maintain the required software for Brant, and I know it's outdated already and would require a new maintainer who can deal with those packages - I already posted about hardware-related maintenance for what it's worth - so it was my first choice: while it meant I had to spend some money, it would have solved my problem and improved Gentoo, even if just for a tiny bit. The price for YubiKey devices is also low enough that, if I felt like providing more FTPS access to customers, I could simply bill it to them without many complaints.

So I went on the manufacturer's (Yubico's) website and tried to buy two of them (one for me to test and set up, and one to give my friend to access the server); despite publishing the prices in dollars, they sell through Sweden and UK, which means they are part of EU's VAT area, and me being a registered business within EU, I should receive a reverse-charge invoice by stating my own VAT ID… never had much of a problem with it, as many of my suppliers are sparse through Europe, I registered for the "foreign-enabled" registry right when I opened business - don't ask me why Italian (and Spanish as far as I can tell) business owners are not enabled by default to have intra-union suppliers.

Now trouble starts: since, as I just noted, not all VAT IDs are valid to use for intra-union trade, there has to be a way to ensure you're dealing with an acceptable party. This is implemented through VIES the VAT Information Exchange System which, for what concerns Italian businesses, only tells you a boolean result of valid/invalid (and not the full registration data that most other states seem to provide). I knew VIES from a previous business agreement, but I never cared much. Turns out though that most e-Shops I encountered validate the VAT ID after order completed ­- or in the case of Amazon it seems like they check their internal database as well as VIES.

Yubico instead validates the request through VIES at the time of registration:

VAT Number could not be validated with VIES at this time. This typically happens when the service is under maintenance. Please retry after some time. For urgent orders, please contact order@yubico.com

Considering that the VIES website has a long disclaimer (which I can't quote here for reasons that will be clear in a moment) stating that they do not guarantee the availability of the service at any time, and only seem to guarantee the validity of the data to the extent that the law ask them to (which probably means "as long as the states' own databases are correct"), relying on such a service for registration is .. bad.

The VIES website is indeed down since at least 11am today (over four hours ago as I write this); for a moment they also gave me an interesting page (which I forgot to save), telling me that there were too many requests' failures from "my IP address" … listing an IP address in the 212/8 range - my actual IP address is in the 94/8 range.

What's the end result here? I'll probably waste some more time trying to get Google Authenticator; Yubico basically lost a customer and a (possible) contributor by trying and failing to be smarter and won't have a dedicated maintainer in Gentoo in the near future. It's sad, because it seems to be easily the most cost- and time-effective solution out there (Google Authenticator is free, but it requires a greater investment of time, and time is money as we all should know).

27 Jan 2012 1:20pm GMT

Seems the patch I committed for the fix was corrupted. So, I am rebuilding and releasing kernels for 3.2 , 3.1 and 3.0.

Thanks for wired for pointing this out. I will be removing the ones from yesterday.

The following kernels now contain the fix:

gentoo-sources-3.2.1-r2

gentoo-sources-3.1.10-r1

gentoo-sources-3.0.17-r2

24 Jan 2012 2:06pm GMT

In my previous installment I ranted about. among other things, the way Rails suggests you to keep a world-writeable log file for the production environment. As I said at the end, I planned on looking at the syslogger gem and that was actually quite helpful.

The idea goes like this: by using syslogger you can tell Rails that the logs have to go through the syslog; in my case that means it goes to metalog, which then filters on the webapp names and pushes it to /var/log/rails, taking care of rotating the log as needed (either due to size or time - the former is quite useful to avoid that rogue bots cause a DoS, which happened to me when I was inexperienced with these technologies!). Of course, this only works on Unix, but that's what I care about anyway.

Beside the placement of the logs, using metalog for me also means I can filter important messages and show them in the important messages' log rather than being just limited to a hidden log file within the app's own tree, and also means that I can mix in the messages of all the running applications, rather than having each report to a different file. If I were to use syslog-ng instead, I could easily make it send the logs via network to another box and aggregate all of them there… but I really don't see the point (yet) for that, and the features that metalog comes with tramp easily the network support.

So how do you achieve this? It's actually pretty easy. Obviously it starts with installing dev-ruby/syslogger (in Gentoo, through Portage, everywhere else, via gem); then you can configure this very easily on both Rails 2.3 and 3.x series (I have one server running Rails 2.3, the other 3.1… I have yet to set up Typo 6.x, but I'll probably do that at some point in the near future, although unlikely before FOSDEM).

The trick is all in config/environments/production.rb, where you have to tell Rails to use a custom Logger; there is already an example, commented-out like that refers to the other gem, SyslogLogger, but you should change it to something like this

  config.logger = Syslogger.new("yourappname")

This way you can distinguish each application's messages in the log. Then in the metalog.conf file you can have:

Rails apps : 
  program_regex = "^(typo|radiant|yourappname)"
  logdir = "/var/log/rails"
  maxfiles = 5
  break = 1

so that everything is then readable as /var/log/rails/current.

I'm not sure how much it impacts performance; I'd be surprised if it decreased them, as metalog also buffers the disk writes, but you never know until you check for sure; in general I still prefer if the (multiple) Rails processes send everything to metalog for my own convenience.

Interestingly, if you have a webapp that does not deal with on-disk files directly, but just with a database, by using syslogger you're basically limiting the writing to the cache directories only, which is probably a positive note.

24 Jan 2012 11:23am GMT

I just released gentoo-sources-3.2.1-r1 for Linux Local Privilege Escalation via SUID /proc/pid/mem .

I plan on creating releases for additional kernels with this patch through the day.

See the link for more info on the privilege escalation.

The following kernel versions contain the patch:

gentoo-sources-3.2.1-r1

gentoo-sources-3.1.10

gentoo-sources-3.0.17-r1

23 Jan 2012 8:33pm GMT

Usually, whenever a new KDE release is published, Gentoo users can update already the same day, as suddenly a complete and polished set of ebuilds appears in the portage tree. (Stay tuned on upcoming wednesday for KDE 4.8.0, it's shaping up very nicely!) How is this possible? Well... let me explain.

If you're a stable version user, you may have never heard of so-called live ebuilds. This is a special variant, usually denoted by a version number ending in 9999, that does not rely on a source tarball. Instead, it contains a URL of a revsion control system (say on anongit.kde.org). When you emerge such a version of a package, the sources of the specified branch are checked out or updated to the newest upstream state, and that is used for building the installation package. Obviously this is not for everyone; depending how well upstream structures commits, things may not build for a while, contain fresh bugs, ... Also, reporting bugs from live versions on Gentoo bugzilla is discouraged as most of the times we can't do anything about it (do it only if you are sure it's a problem with the ebuilds, not with the source). If you're running live, you should be willing to hack yourself and work with upstream.

However, many of the Gentoo KDE team members run these live ebuilds, partly the current bugfix branch (i.e. KDE/4.8), partly even git master. They continuously keep the live ebuilds in the Gentoo KDE overlay updated to the newest state of the source. When a release is made, the corresponding live ebuilds of this branch are copied to the version ebuilds. For example, the KDE/4.8 branch live ebuilds have the version number 4.8.49.9999 (i.e.
kde-base/kdelibs-4.8.49.9999), so when the pre-release tarballs for KDE 4.8.0 were released to the packagers a few days ago, we only had to copy all 4.8.49.9999 ebuilds to 4.8.0 and immediately had a working set for testing. Most problems at that point are only caused by changes in tarball packaging. As distribution packagers get the pre-release tarballs (that still may change due to last-minute bugfixes) a week before the official release date, these can easily be fixed in time.

This also means that KDE maintenance in Gentoo is really a team effort. Whoever moves a released version to the main portage tree and/or commits bugfixes there builds on all the work that the team has done in the overlay in the meantime. Cheers!

22 Jan 2012 5:43pm GMT

You might or might not remember my fighting with mod_perl and my finding a bug in the handling of logs if Apache's error log is set to use the syslog interface (which in my case would be metalog). For those wondering the upstream bug is still untouched goes without saying. This should have told me that there aren't many people using Apache's syslog support, but sometimes I'm stubborn.

Anyway, yesterday I finally put into so-called "production" the webapp I described last week for handling customers' computers. I got it working in no time after mongoid started to behave (tests are still restricted, because a couple fail and I'm not sure why - I'll have to work on that with the next release that require quite fewer hacks to test cleanly). I did encounter a nasty bug in "best_in_place"http://rubygems.org/gems/best_in_place which I ended up fixing in Gentoo even though upstream hasn't merged my branch yet.

To get it in "production" I simply mean configuring it to run on the twin server of this blog's, which I've been using for another customer as well - and got ready for a third. Since Rails 3.1 was already installed on that box, it was quite easy to move my new app there. All it took was installing the few new gems I needed and…

Well here's the interesting thing: I didn't want for my application to run as my user, while obviously I wanted to check out the sources with my user so that I could get it to update with git … how do you do that? Well, Passenger is able to run the application under whatever user owns the config/environment.rb file, so you'd expect it to be able to run under an arbitrary user as well - which is the case, but only if you're using version 3 (which is not stable in Gentoo as of yet).

So anyway I set up the new passenger to change the user, make public/assets/ and another directory I write to group-writable (the app user and my user are in the same group), and then I'm basically done, I think. I start up and I'm done with it, I think… but the hostnames tell me that "something went wrong", without any clue as to what.

Okay so the default for Passenger is to not have any log at all, not a problem, I'll just increase the level to 1 and see the error… or not? I still get no output in Apache's error log .. which is still set to syslog… don't tell me… I set Passenger to log to file, and lo and behold it works fine. I wonder if it's time for me to learn Apache's API and get to fix both, since it looks like I'm one of the very few people who would like to use syslog as Apache's error log.

After getting Passenger to finally tell me what's wrong, I find out both the reason why Rails wasn't starting (I forgot to enable two USE flags in dev-ruby/barby which I use for generating the QR code on the label), but I also see this:

Rails Error: Unable to access log file. Please ensure that /var/www/${vhost}/log/production.log exists and is chmod 0666. The log level has been raised to WARN and the output directed to STDERR until the problem is fixed.
Please note that logging negatively impacts client-side performance. You should set your logging level no lower than :info in production.

What? Rails is really telling its users to create a world writeable log file, when it fails to write to it? Are they freaking kidding me? Is this really a suggestion coming from the developers of a framework for Web Applications which should be security-sensitive? … Okay so one can be smarter than them and do the right thing (in my case make sure that the log file is actually group-writeable) but if this is the kind of suggestions they find proper to tell you, it's no wonder what happened with Diaspora. So it's one more reason why Rails shouldn't be for the faint hearted and that you should pay a very good sysadmin if you want to run a Rails application.

Oh and by the way the cherry on top of this is that instead of just sending the log to stderr, leaving it to Passenger to wrangle - which would have worked out nicely if Passenger had a way to distinguish which app the errors are coming from - Rails also moves the log level to warning, just to spite you. And then tells you that it impacts performances! Ain't that lovely?

Plan for the day? If I find some extra free time I'd like to give a try and package (not necessarily in this order) syslogger so that the whole production.log thing can go away fast.

22 Jan 2012 8:19am GMT