After reading http://groveronline.com/2013/06/fedora-for-servers/ and the comments on lwn ( https://lwn.net/Articles/552800/ ) I thought I would share some thoughts on this. First, I agree with Andy that this is a good way to use Fedora instances. With so many ways to spin up quickly a instance that does just exactly what you need it to these days, this should work great for a number of use cases. Lennart points out that containers is a even quicker/more minimal way to do things for some use cases ( https://plus.google.com/115547683951727699051/posts/G1DTXebUGGN ). Tie this to a CM like ansible and it's easy to fire up and test your app in the next Fedora and move to it when you are ready, or even in an updated Fedora of the same version you are currently running. Some of the comments on the lwn article in particular reiterate a strange divide that I see in the open source world. This group of folks wants a platform where they can exactly specify every version of everything they use and never have to change those versions based on external factors. To this group spinning up a Fedora instance to run their app on isn't that appealing because Fedora the distribution only offers usually single versions of packages (or perhaps some compat packages for backward compatibility in some cases), so they have to manage their own versions/updates/setup. For them the platform itself should be only whats needed so they can bring up their own stack of tools. Thats a perfectly valid use case, and perhaps one Fedora should target some more. That said, I'm still not sure I understand why there are so many folks who wish to design/build/maintain applications that way. Is this a fallout from many projects/people using "agile" development? Perhaps it shows my age, but what ever happened to designing an application such that it used interfaces other things it needed exposed and working with those other projects to make sure those interfaces were still stable over time or porting to new ones was possible/not too difficult? Why wouldn't you want to leverage the distributions packaged versions of projects you depend on? Sure, you have your app working on Fedora 18 and then in Fedora 19 libfoo is libfoo2, so you keep using Fedora 18 until you have ported your app to use libfoo2 and then spin up Fedora 19 test instances and test until you are sure your app is working right. If libfoo2 dropped some functionality you needed, could you not ask them to readd it or add some new way to do the same thing? The trade offs here seem poor to me, but of course everyone will do things the way they feel is best for their use case. I have a feeling that applications developed with a 'ship my exact versions' will more and more start seeing the issues in this model as time goes on, but we will see.

Another very dull week in rawhide. No particular breakage or issues here. I've been on the latest 3.10 rc3 kernel and since it's an RC the debugging is off, so its been nice and fast. I did play around with dnf a bit. dnf is the next generation yum that is being worked on (likely won't become yum until f21-22 or so, it's still missing a lot of features). It seems to mostly work ok, but I am not sure I've seen much speed difference between it and yum and it seems to use more cpu at some tasks. Looking forward to it improving more.

Another week of rawhide. Only one issue really stands out from this week: The kernel switched to using a script provided by systemd to install. See: http://jwboyer.livejournal.com/47131.html for the low down. There was a slight mistake made in the script however, which meant that rawhide users wouldn't see new kernels appear in grub.conf. This has been corrected today, and if you update to at least systemd-204-4.fc20 it should be fixed. You may need to re-install old kernels or manually add them to grub2.cfg if you want to boot them. Note that this does not affect Fedora 19 Branched, those kernels still use the old scripts and should work fine. In other news Fedora 19 Beta went out today. A pretty smooth and uneventful release day. Please do download it and try it out and report bugs on it. F19 is shaping up to be a nice release, IMHO.

I got to thinking recently: Why are interactions with some small number of community members always unpleasant? It seems to me as a general rule that the people who are unpleasant to deal with (or at least all of those I can think of in any of the communities I am involved with) all have the same basic fundamental viewpoint: Assume malice by default. By this I mean that when they see something and post/tweet/mention it, they approach it from the idea that the community or parts of it are deliberately conspiring, hiding, planning, or doing the thing with "mallice aforethought", and that the community or parts of it must PROVE that they are not doing so. There also seems a subset that assume some kind of malicious ignorance ("You should google this, since you didn't you are stupid and wasting my time"). In contrast the vast majority of people in communities I am involved in either assume nothing ("hey, whats up with this, can you tell me?") or assume good faith ("This looks like a bug, but perhaps I don't understand what you are trying to do here, can you explain?") and are generally quite pleasant to deal with. I'm sure there are counter examples, but I wonder if in general people agree?

So, another pretty quiet week overall. There was a soname bump in libpng, but it was untagged and never landed in a rawhide compose. The maintainer is going to create a compat package for the old version first, then update the main package so things can slowly move to the new version over time instead of breaking things over many days. I'm glad people are pushing back on these kinds of things to keep rawhide more generally useful. There are of course still unannounced soname updates that do break things, but I get the impression we are slowly getting better on this. Another related nice change that landed this week is that rpm by default is only going to look for dependencies in .so files that also start in 'lib' or 'ld' and not just anything ending in .so. So far I've not seen any breakage due to this change and it should nicely reduce the number of deps carried in the collection. Finally, something to note about rawhide testing that I am not sure I have any kind of answer for, but I think is worth exploring: When running rawhide it's generally a good idea to reboot frequently. New kernels arrive almost daily, and it's good to check that things all still boot normally (this exercizes at least: dracut, kernel, grub2 or other bootloader, systemd, other stuff that starts only on boot, etc). However, this also means that there's likely not very many people running rawhide that test longer uptimes. This would exersize things like: updates of long running processes, systemd, memory leaks, logrotation, and other things that collect up and don't run at boot). I've hit several odd systemd issues on upgrade, and I'm sure there's more out there. Perhaps we should just say that this is what branched is better suited for (as kernels and reasons for update are less frequent there), or perhaps there's some artificial way to test this, or a vm or the like could be left running to do so. Just something to consider.

Another week of rawhide has gone by, so another blog post. ;) Utterly boring this week. I cannot recall any problems here at all. In branched f19 news, we are moving into Beta freeze today. You can follow along on blockers and freeze exceptions (or propose new ones) at the blockerbugs application. There was a bunch of activity from various desktop sigs in the last week to get their spins down to their target sizes, but I think we are in pretty good shape on that now. Hopefully it will be a smooth and on-time Beta release for once.

Have you submitted a talk? Fedora is doing a new conference this year: flock It's setup more like a traditional conference (sessions proposed in advance) and it's timed after Fedora 19 comes out, but early in the Fedora 20 cycle, so there's good ability to plan things for the very next Fedora at it. Also, hopefully lots of folks from around the world will be able to attend. (Next years flock might well be in Europe). I've submitted one talk so far that I thought might be fun an informative:   The life of a Fedora package: "This talk will follow in detail one package through all the steps of it's life in Fedora and explain what happens at each step and all the systems and applications that deal with packages. Starting from idea, to project, to packaging, to review, importing, building, update, signing, mashi ng, syncing, testing, maintaining, retiring and end of life" As usual, my problem isn't what to talk about, but that there are too many things to talk about. :) I've considered a "Intro to Fedora infrastructure" talk, but I'm not sure how interesting people would find it. I also submitted a 'sprint': Highly available databases: Fedora infrastructure currently depends on serveral database servers, we want to come up with a plan to make them highly available and no longer a single point of failure. postgresql wizards wanted to help us with this sprint! I'd love to get rid of having to schedule outages when we reboot or work on our database servers. :) Anyhow, I might well propose some more sprints/talks/hackfests/workshops before the deadline (May 31st). If there's something that you would particularly like to see me talk on or run a workshop/hackfest/sprint on, drop me a line and I might be willing to. :)

After reading about upcoming bind10 changes and never really being all that happy with named, I decided to look around at other solutions for DNS serving. I've been running unbound (along with dnssec-trigger) on my laptop for a while, so I was pretty sure it could meet my needs for a recursive/caching solution, but I hadn't found anything to replace authoritative servers until nsd was pointed out to me. Replacing my authoritative server with nsd was pretty easy overall. I could reuse the same zone files, I just had to add a bit of nsd config to load them. My main firewall/gateway machine at home was also functioning as a caching server for machines here, so I had to replace named with nsd only listening on the external interface, and unbound only listening on all the internal interfaces. Worked like a charm. nsd now answers external queries for my domains and unbound does caching for other internal machines. My main internal webserver/mail server machine also was running named and serving some internal only zones. nsd doesn't have any kind of 'views' setup like named has so I had to ponder on this for a bit until I noticed that unbound can also optionally have local data. I just setup my zones there in unbound as local-data and it then was able to do what I needed (basically I need my domains to resolve to the internal ip's on that machine so apache is happy, etc). It can also easily resolve *.example.com to whatever single ip you like. Very handy. The only downside of this is that dnssec doesn't work right with local-data. Perhaps if I signed the zones with the same keys at the external site it could, but not sure it's worth the trouble. Unbound also has a nifty control program (non surprisingly called unbound-control) that can let you do things like flush cache for just a single specifc zone or host, setup a forward over a vpn or other link for just _some_ domains, etc. It's really nice and flexable. Both unbound and nsd as very dnssec aware and will try and validate as much as possible. Also, nsd has rate limiting built in so it's resistant to DOS or DDOS amplification attacks. So, if you are looking to replace bind, nsd seems to work nicely for authoritative and unbound does great for recursive.

So, this last week in the world of rawhide: The 3.9.0 kernel came out, and quickly was replaced with 3.10.0 git snapshots. So far they have been all running just fine here, although there was a few there that had some problems with wired ethernet. Xfce 4.10.1 updates came out. These are just bugfix/translation updates for the existing 4.10.x Xfce stable stream. The more important fixes we were already carrying as backported patches, so there's nothing too thrilling here. Updates are out for f18 and f19, please do karma them up if they work for you. Rawhide had the packages monday. There was some yum breakage last thursday. Luckily for me I didn't update early in the day so I saw the reports and was able to update the fixed yum from koji before applying my daily updates. This is another nice way to be on rawhide and a bit more conservative, just wait and update late in the day so others will hit the big problems before you do and provide solutions. (It's also another great reason to be on the test list to see those kinds of things). Ran into an odd dig core dump issue (try: "dig fedoraproject.org +sigchase +topdown" and see the cores pile up). Unfortunately, abrt didn't want to file it for me, so I will have to do a bit more digging and make sure it's filed. A fedora community member on g+ whipped up (pun intended) a rawhide wallpaper thats pretty cool: https://plus.google.com/116680320767812371369/posts/JTy5m3YhPAJ is the post. Until next week... move em up.

A final rawhide post for April. Another pretty dull week here in rawhide land. The updates flow has been pretty heavy, but breakage has not been. There's a very likely fix to a xfce4-session crash that some folks had been seeing in f19/rawhide. It was quite an anoying bug to deal with (you would be using your desktop and then, bam, back to lightdm). I really hope it's fixed. If not, please do file it or note it on one of the open bugs. Kernel 3.9.0 final was released, and rawhide has moved on to 3.10. :) Looking forward to some more quiet weeks.