This post is a day late, but we had Fedora 18 release yesterday, so I was kinda busy. ;) So, another week of rawhide. This one was a bit more rough than the past two, but still nothing too bad IMHO. Issues:

• There was a libselinux update that pushed out that broke X/udev. Happily it was discovered by Dan Walsh early that morning so we could send out an email to lists telling folks not to update that version. It was then fixed later that day, so only one days rawhide had the bad build in it. Lesson: Always be subscribed to the devel/test lists to see issues like this and read them before doing your daily update.
• colord had an update that was pulling in tons of 32bit packages on my 64bit laptop install. This one turned out to be a bit tricky to fix. The problem was that it was trying to obsolete some old noarch packages, but yum saw both the 64bit and 32bit colord in the repos and picked the 32bit one to do the obsolete. This was fixed the next day by Kalev Lember. He moved the colord libs to a libs package so it no longer needed to have both arches in the 64bit repo. Rawhide had the bad build in for one day again, and it was easy to exclude.
• iwlwifi regression. When I moved my laptop to the 3.8.x pre kernels, my wireless slowed way way down. (backups that normally take 1hour started taking 12). This was already filed as a bug and someone from intel was on the case. A one line patch and things are much better here. This is bug 863424. Fixes hopefully will be merged soon upstream.
• Xfce4-session started crashing on me a lot. I haven't tracked this fully down, but wiping my session cache seems to have completely cleared it up. So, somehow I had something very odd in my session cache.

Otherwise, everything has been going along fine. If these are the sorts of problems you can handle, seems like rawhide should be fine for you to use. ;)

Fudcon Lawerence 2013 is coming up ( see: http://fedoraproject.org/wiki/FUDCon:Lawrence_2013 ) next week. Fudcon's are always a great time. You get to have high bandwith discussions with tons of smart people and hear about all kinds of cool things they are working on and share some food and laughs. I find that it's very easy to get pulled in many directions and sidetracked by other people's new shiny things and forget about the things you wanted to discuss / get done. To help with that, I thought I would write up my list of things I would really love to get discussed / work on in hopes that can remember them while I am there. :)

1. Fedora Infrastructure will of course be there in force. I'd like to see if we can plan out what things we want to get done in 2013 from a high level view. I'd also like to decide on a topic for a FAD later in the year to knock out something we haven't been otherwise able to get done. Saturday morning we are going to try and gather and plan what we want to work on when, etc. Please join us. If you have any infrastructure questions or concerns, feel free to bend my ear anytime. We have: https://fedoraproject.org/wiki/Fedora_infrastructure_tasks_2013 as an ideas container.
2. I want to brainstorm on my ideas around Fedora Formulas (name subject to change): https://fedoraproject.org/wiki/Fedora_formulas I have signed up for a tech session slot on Friday to do just that.
3. I want to try and discuss and come up with some kind of consensus around some EPEL issues that we have just not decided on: What channels will EPEL not overlap with and What to do with incompatible upgrades. I've also signed up for a friday session on this. I'll try and gate IRC folks into the discussion too for those that can't make it. Greg Swift setup an initial wiki page on this: https://fedoraproject.org/wiki/EPEL/Rethinking
4. I have some ideas around rawhide improvement that I intend to run by the list later in the week... If there's enough interest in a session around that I can do one, but I think we can mostly just discuss those things on list/IRC.

For you other Fudcon goers... what do you plan on getting done or discussed? What is exciting you about the currently listed talks / sessions?

Well, another week with rawhide full time on my main laptop. The volume of new packages has picked up since maintainers are back from holidays, but overall still very usable day to day. A few things to note however: There was some selinux policy / libsepol updates yesterday and somehow things got reset to enforcing mode (I have been running permissive due to firewalld and denials around my mail setup). I don't see any commits/changes to the scriptlets on those packages though, so I am not sure what happened. If anyone else sees this perhaps we could try and track it down. New systemd 197 came in and works just fine. No problems that I can see so far. My wlan0 is now wlp3s0, but I could care less what it's called. Kernel 3.8.0-0.rc2.git2.2.fc19 came in and is working just fine so far, although I got a btrfs warning in dmesg. Will see if it's worth filing, although it's not caused any issues. There was a libcdio abi bump, but the maintainer properly announced it, then took care of rebuilding the affected packages. Due to some bad timing all the builds didn't land in one rawhide compose, but they cleared up the next day. Kudos to them. ;) There's currently a weird / bizzare debuginfo bug preventing xulrunner from building, so firefox updates might be lagging, and there's currently a broken dep there, but --skip-broken works around it fine. zsh updates result in some spewage on my shells. It's a difficult problem due to the versioning of functions (you update and the old version thats running has it's functions dir removed, the new one works fine after execing). I'm not sure there's an easy fix here, but I filed a bug for the zsh maintainers to look at. So, all in all, things are doing just fine here day to day in the rawhide world still.

There was a discussion on the Fedora devel list late last year about moving to a more rolling release model, and I have seen people continue to suggest this in various places since then, so I thought I would write up my thoughts on rolling releases and how they can/could apply to Fedora. Note that I am speaking only for myself here, not my employer, FESCo or anyone else. First, lets talk terminology. What is a "rolling release"? In reference to Linux distributions, it's where you provide a continual stream of updates for your distro and all work is done in one branch (there's no seperate point releases, there's only the distro). (See: http://en.wikipedia.org/wiki/Rolling_release for much more discussion and terms, but I think that the above boils it down some). Now a bit of Fedora background: Fedora has a devel/master branch, called (for historical reasons) rawhide. Is rawhide a rolling release? I would argue that it is. In addition currently Fedora has at various times 2-3 more branches/releases being maintained. Updates to these are guided by the Updates policy: http://fedoraproject.org/wiki/Updates_Policy. These releases are on a 6month (roughly) cycle. A new branch/release is branched off rawhide, then stablized and released. Old releases are retired 1 month after 2 releases happen. These are not at all rolling releases by the above definition. So, lets take some updates examples and see how things work now in Fedora vs what they would look like in a rolling distro: You have foo-1.0-1 installed and foo-1.1-1 comes out. It's minor bug fixes, possibly docs and translation updates.

• On supported stable Fedora releases: You would possibly get this update if the maintainer felt the bug fixes were worth bothering with.
• On a rolling release/rawhide: You would get this update and not really notice it unless you were hit by a bug in this package or needed the docs or was heavily involved with it.

You have foo-1.0-1 installed and foo-2.0-1 comes out. It's a major new version. The UI changes along with lots of other things.

• On supported stable Fedora releases: You shouldn't get this update if at all possible, the exceptions might be if upstream stopped supporting foo-1.0 at all and this fixed security updates that couldn't otherwise be fixed any other way. If you are a happy foo-1.0 user, I would argue that you don't want foo-2.0 to suddenly appear and cause you to have to learn how to use it. In a timed release model like Fedora currently uses, you would get foo-2.0 at the next Fedora release, when you have time and desire to sit down and learn all the new changes. You can schedule it on your timeline, not when it appears in the repos.
• On a rolling release/rawhide: You get this update quickly, and must deal with learning the new UI right then. If you have important work in foo-1.0 and are on a deadline, you don't have much choice when you have to learn 2.0.

You have libfoo-1.0-1 and libfoo-2.0-1 comes out. This is a major new library version, the ABI/API changes. All applications using this must rebuild and probibly get patched to work with the new version.

• On a supported stable Fedora release, you won't see this update. You will get it at the next Fedora release. This gives you time to port your applications to the new ABI/API, and you can sit down and fix things when you upgrade to the next Fedora release.
• On a rolling release/rawhide: You get this update pretty quickly. The majority of things that need rebuilding are done, but stragglers are sometimes left broken by the upgrade. If you use one of them you are out of luck, or must spend time getting them working as soon as the bundle of packages land in the main repo. There's some ways around this of course, like compat packages for things, but in general if your application is very slow moving there may be a lag in support.

You have rpm-N and new rpm-N+1 is released with a new package format/compression/payload. (Or you move all / dirs to /usr, or...)

• On a supported stable Fedora release, you never see this update. It's only on the next Fedora release.
• On rawhide/rolling distro: You announce this in advance and try and get people to make sure they upgrade to the new version, you add docs about the error the old version prints when it can't deal with the new version. You have a flag day to cut over and anyone who missed the memo has to go figure it out before they can get more updates.

(I could go on with the examples, but I could do that all day. Feel free to suggest other scenerios in comments ;) As an aside there's some notable exceptions to the Fedora updates policy worth mentioning:

1. The Linux kernel. Currently Fedora keeps pretty close to the most recent upstream stable kernel version for _ALL_ Fedora stable releases. This occasionally causes problems for folks as new bugs make a machine that was working fine not boot, or have issues. There are a number of reasons for this: First, backporting security and bugfixes to old kernel versions is a vast amount of work. Fedora has a crack ninja team of kernel maintainers, but they are a small team. Backporting new hardware enablement would also be very difficult, and new hardware is always appearing. Not allowing new kernels on old releases would result in needing to upgrade to support new hardware, which isn't very nice. Additionally, kernel is a special package in Fedora, in that it's one of the few you can install multiples of. So, even in the event of some new bug, you can boot the older working kernel and debug the new problem.
2. KDE. KDE upstream unfortunately has releases that are not at all aligned with Fedora. This is of course their choice, but it makes things hard for us. Additionally, older major versions aren't supported, so, when Fedora N ships KDE 4.x at release time and 4.x+1 comes out in a month, we really have no choice but to update to that as again backporting manpower isn't there.

So, do I hate rolling releases and want nothing to do with them? No. I think rolling releases are quite valuable, and a good fit for a small group of folks. You may like/want/use rolling releases if:

• You are a early adopter (You WANT foo-2.0 as soon as it's announced)
• Love troubleshooting issues (You don't mind when things break and are happy patching something to be fixed up for a new version, or reporting bugs on a new version of something)
• Are an integrator/software developer. (You want the latest libraries and toolchains to make sure your project builds and works there and takes advantage of the latest cool features)
• You are a distro integrator and enjoy making all the parts of things fit together as soon as your upstreams have released them.

However, if you are anyone else, you probibly don't want to be using a rolling release. With a timed release, YOU can choose when you have time (+ or - 7 months in Fedoras case) to upgrade to the newest collection of software, relearn new applications and ui's, rebuild/fix local code to new libraries, etc. With a rolling release, you are at the mercy of the upstream projects and your distro as to when you have to accept and adjust to a change. So, if rawhide is a rolling release and some folks want Fedora to do one, why don't they just use rawhide? I think there's a number of reasons for that, many around rawhide being too raw for day to day use. However, I have been working on a number of ideas to improve day to day usefulness of rawhide, as well as using it myself day to day. Look for those ideas soon. Possibly we can address some of those concerns and get more folks using rawhide day to day and have the best of both rolling releases and timed stable releases.

My main laptop has now been on rawhide for a week (yes, I installed it christmas day :) Overall things have been pretty quiet, which may well be due to the holidays and a lower rate of flow of new rawhide packages. I have run into an anoying crash in xfce4-session, but otherwise everything I use day to day works fine (General Xfce, hexchat, claws-mail, midori). (crash is bug 891113 and filed upstream as 9709 ) I've been pondering ideas on how to make rawhide in general more stable and usable day to day over the holidays. I'll likely open a discussion on the devel list soon with those ideas. Hopefully we can look at implementing some of them soon.

With the roll out of communities on google plus, I made a "Fedora Project" community for any g+ users to join: Fedora Project google plus community If you use google plus, do join us. Note the topics on the left and make sure to put your posts into the approprate topics. I think this might be a nice communications channel, althougth there are still rough spots, like the fact that community posts flood your main stream instead of staying in the community tab. ;(

Another early day here... time for a quick breakfast with folks who are still around and then head to the airport. Long day of traveling and I seem to have picked up a cold/flu or something along the way. ;( Finally made it home happily. Tomorrow, I'm going to work on docs for our 2 factor auth and look at enabling it in production. :)

Another far to early a day and we were over hacking on two factor auth again over at the Red Hat tower. (On the 13th floor this time!). We got things further deployed in staging, got the config all puppeted, got the provisioning (thanks to totpcgi) setup and working. Got some ssl cert CA and bundle issues worked through (pam_url verifies it's connection to the cgi, and the cgi also verifies that pam_url has a valid client cert). Ran into an issue where 32bit machines were sending the incorrect username. It was just garbage. ;( Some great community effort and we managed to track down the bug in the pam_url module c code, then fixed up some cert issues (the server wasn't able to use a server cert to auth against itself). Everything seems now to be working in staging, and we will push it out live friday most likely.

First day of the FAD, and a very productive one indeed. ;) First off got things pushed out to make Fedora 18 Beta release live in the morning and everyone happily downloading. Next we all met up in the hotel lobby and walked over to the Red Hat tower where we were provided some lovely meeting space by Red Hat. We discussed policy and plans and then dived in to implemented. After a long day of puppet commits, packaging and debugging we had 2 factor auth working in our staging env! Tomorrow (and some more tonight) we are going to work on adding yubikey backend and work on the enrolling UI/cgi interface. As a high level overview, we are using pam_url (now packaged and accepted in Fedora/EPEL) on client machines and totp-cgi (packaged and under review) as the server end. Then we are adding some code on to support yubikeys. Down the road we may well look at pulling totp-cgi functions into FAS directly, but we decided that we want to get things working now. This has been a great group of folks to work with... dividing things up and getting things done. ;)

Long day of travel... but I made it to Raleigh. ;) My shuttle ride came super early, but it turns out it was good, as I was the first person picked up and barely got to my plane in time, which turned out to be about 20min late leaving. :( That got me to Houston about 30min late, which gave me a very short time to run across the airport and just barely catch my plane to Raleigh. On arrival I met smooge and waited a bit and met mricon. Hit the lovely hotel and checked and then got dinner with folks... now just getting ready for the release tomorrow and chatting with folks in the lobby. All in all a good start! :)