Want to know what's really going on with your server? Then you need to know these essential commands. Once you've mastered them, you'll be well on your way to being an expert Linux system administrator.
Depending on the Linux distribution, you can run pull up much of the information that these shell commands can give you from a GUI program. SUSE Linux, for example, has an excellent, graphical configuration and management tool, YaST, and KDE's KDE System Guard is also excellent.
However, it's a Linux administrator truism that you should run a GUI on a server only when you absolutely must. That's because Linux GUIs take up system resources that could be better used elsewhere. So, while using a GUI program is fine for basic server health checkups, if you want to know what's really happening, turn off the GUI and use these tools from the Linux command shell.

I know we’ve been covering Puppet Labs modules for the last few weeks, but I’d like to specifically call out all the great community contributions we’ve had on this module. In fact, this module started out as a community members project, was forked by someone at Puppet Labs, modified and later enjoyed significant contributions from other community members.

We deploy a lot of code. Deployinator is our creation to make that as easy and painless as possible. Deployinator is a one button web-based deployment app. Hit that button and code goes to our webservers and is serving requests in almost no time. Using Deployinator we’ve brought a typical web push from 3 developers, 1 operations engineer, everyone else on standby and over an hour (when things went smoothly) down to 1 person and under 2 minutes.

At Etsy, we’re doing what’s come to be called Continuous Deployment. However, what we’ve learned is that having a tool like Deployinator is useful for more than just enabling that. This post is about those benefits – for anyone deploying web code.

Why

Our job as engineers (and ops, dev-ops, QA, support, everyone in the company actually) is to enable the business goals. We strongly feel that in order to do that you must have the ability to deploy code quickly and safely. Even if the business goals are to deploy strongly QA’d code once a month at 3am (it’s not for us, we push all the time), having a reliable and easy deployment should be non-negotiable.

Martin Englund posted an open question to the Puppet mailing list a few days ago asking how people are verifying their systems are built as expected: When you write code, you always use unit testing & integration testing to verify that the application is working as expected, but why don't we use that when we install a system? What are you using to verify that your system is correctly configured and behaves the way you want? He linked to a blog post demonstrating how he was verifying his machines using Cucumber. Coincidentally, about a week earlier at Devopsdays in Gent, I was talking to Felix Kronlage and Bernd Ahlers from bytemine about doing similar things through testing SSH and mail delivery with cucumber-nagios. It's pretty cool people are thinking about doing BDD/TDD with infrastructure, and it's even cooler that the tools are at the point where doing this is actually possible. When doing software testing, your testing tool is normally separate from the language and libraries you're building the software with (but almost always written in the same language). When testing your infrastructure, I think it makes perfect sense to apply this practice.

Collection of Test Driven Infrastructure Links

This pages shows a list of links i've collected related to the topic(s) of

behavior driven infrastructure
test driven infrastructure
test driven administration
test driven deployment
monitoring driven infrastructure

Mac Sys Admin 2011 – European Macintosh System Administrators Meeting 2011 –

You’ve bought Pro Puppet, downloaded a couple of modules from the Puppet Forge (and have written some of your own too), and you’re on your way to implementing your Puppet environment when it hits you: something feels bulky with the way you’ve designed your Puppet code. Your modules may not be portable between environments (development, testing, production) without significant tweaks, each of your node declarations may require a number of variables in order for the code to work, or you’re constantly needing to open up your modules to account for changes in your environment.

List of statsd server implementations

Statsd is a simple client/server mechanism from the folks at Etsy that allows operations and development teams to easily feed a variety of metrics into a Graphite system. For more info on statsd read the seminal blog article on Statsd “Measure Anything, Measure Everything”.

As would be expected there are statsd clients in many languages. But, there are also many implementations of the statsd server. This is nice because each organization can pick the one that best fits them. For example, a python shop might prefer to deploy a python based statsd instead of Etsy’s original node.js implementation. Also, there are some statsd implementations that diverge from the original design and provide additional features.

logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs.

It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.

Utilities, tools, and scripts for managing and tracking a fleet of Macintoshes in a corporate environment.

There is NO reason, not to use a version control system while developing puppet manifest/modules. Stating that should be an open door. It allows you to go back in time, share things more easily and track your changes. There is a lot of information out there on how to work with git or any other system. But here a few tips that might help you developing modules:
Tip 1 : give each module it's own repo and use a superproject to join

In a lot of blogposts and even in the excellent Pro Puppet book I see people checking in their entire environment directory into version control.

I'm all for version control but if you manage your modules dir as one flat repository, you loose the way to easily update and share modules from the forge. In essence you are doing a copy that starts living it's own life.

Over the last year I haven't only grown very fond of coffee, but also of infrastructure. Working on Scalarium has been a fun ride so far, for all kinds of reasons, one of them is dealing so much with infrastructure. Being an infrastructure platform provider, what can you do, right?

As being responsible for deployment, performance tuning, monitoring, infrastructure has always been a part of many of my job I thought it'd be about time to sprinkle some of my thoughts and daily ops thoughts on a couple of articles. The simple reason being that no matter how much you try, no matter how far away from dealing with servers you go (think Heroku), there will always be infrastructure, and it will always affect you and your application in some way.

On today's menu: monitoring. People have all kinds of different meanings for monitoring, and they're all right, because there is no one way to monitor your applications and infrastructure. I just did a recount, and there are no less than six levels of detail you can and probably should get. Note that these are my definitions, they don't necessarily have to be officially named, they're solely based on my experiences. Let's start from the top, the outside view of your application.

At Boston DevOpsDays 2011 last week I hosted an open spaces discussion during which we prognosticated on what the everyday sysadmin would look like in 10 years time.

A lively discussion followed and out of it we came up with a few key predictions that we all loosely agreed on; the future sysadmin will:

Write code.

We all agreed that there’s little place for a future sysadmin that can’t (at the minimum) write scripts, and nominally write and understand code in a non-shell programming language.

Do a lot of data analytics.

We thought that any future sysadmin will be much more of a data-driven engineer; they’ll build systems based on engineering not gut feelings or “because it worked last time”. The future sysadmin can do math because the future sysadmin does more science.

Work on a higher level of abstraction.

The future sysadmin needs to build complex systems by treating what we now think of as systems as building blocks. They’ll not think as much about network ports, IP addresses and machines but instead think about the interactions of applications and instances of those applications.

Whenever I need to setup a new service on one of my hosts, the first thing I do is head to the forge and GitHub to try and find a decent Puppet module that already exists for it.

I almost always leave in disappointment.
Puppet modules are libraries

Much like string.h provides everything you need to manipulate strings in C, your Puppet modules should provide everything needed to manage a service out of the box. By that I mean, I want to pull down your module to enable the functionality I need in Puppet without modifying your module at all.

Introduction

While there is not one true definition of devops (similar to cloud), four of it's key-points resolve around Culture, Automation, Measurement and Sharing (CAMS). In this article we will show how this affects the traditional thinking of the sysadmin.

As a sysadmin you are probably familiar with the Automation and Measurement part: it has been good and professional practice to script/automate work to make things faster and repeatable. Gathering metrics and doing monitoring is an integral part of the job to make sure things are running smoothly.
The pain

For many years, operations (of which the sysadmin is usually part) has been seen as an endpoint in the software delivery process: developers code new functionality during a project in isolation from operations and once the software is considered finished, it is presented to the operations departement to run it.

During deployment a lot of issues tend to surface: some typical examples are the development and test environment not being representative to the production environment, or that not enough thought has been given to backup and restore strategies. Often it is too late in the project to change much of the architecture and structure of the code and it gives way to many fixes and ad-hoc solutions. This friction has created a disrespect between the two groups: developers feel that operations knows nothing about software, and operation feel that developers know nothing about running servers. Management tends to keep those two groups in isolation from each other, keeping the interaction at the minimum required. The result is a 'wall of confusion'

Puppet is a wonderful system automation tool, however the learning curve can be a little steep. We've collected some of the errors messages and "strange" behaviour you may come across together with explanations to help overcome these hurdles and boost adoption of this fabulous tool.

If you have any useful errors and explanations, please do send them in and we'll update this article.

Blueprint
Blueprint reverse-engineers servers

Easy configuration management.
Detect relevant packages, files, and source installs.
Generate reusable server configs.
Convert blueprints to Puppet or Chef.
No DSLs, no extra servers, no workflow changes.

Blueprint looks inside popular package managers, finds changes you made to configuration files, and archives software you built from source. It runs on Debian- and RPM-based Linux distros with Python >= 2.6 and Git >= 1.7.
Blueprint I/O moves blueprints around

Centralized configuration management.
Export and backup server configurations.
Push and pull blueprints anywhere.
Bootstrap servers painlessly.

Blueprint I/O pushes and pulls blueprints to and from a Blueprint I/O Server, making it easy to use blueprints anywhere. DevStructure provides a free Blueprint I/O Server at https://devstructure.com, which stores blueprints in Amazon S3. Alternatively, you can build your own backend server implementing the Blueprint I/O API.

collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files.
What does collectd do?

collectd gathers statistics about the system it is running on and stores this information. Those statistics can then be used to find current performance bottlenecks (i.e. performance analysis) and predict future system load (i.e. capacity planning). Or if you just want pretty graphs of your private server and are fed up with some homegrown solution you're at the right place, too ;).

Usually one graph says more than a thousand words, so here's a graph showing the CPU utilization of a system over the last 60 minutes:

Reposado, together with Python, the "curl" binary tool and a web server such as Apache 2, enables you to host a local Apple Software Update Server on any hardware and OS of your choice.

Reposado contains a tool (repo_sync) to download Software Update catalogs and (optionally) update packages from Apple's servers, enabling you to host them from a local web server.

Additionally, Reposado provides a command-line tool (repoutil) that enables you to create any arbitrary number of "branches" of the Apple catalogs. These branches can contain any subset of the available updates. For example, one could create "testing" and "release" branches, and then set some clients to use the "testing" branch catalog to test newly-released updates. You would set most of your clients to use the "release" branch catalog, which would contain updates that had been through the testing process.

radmind - A suite of Unix command-line tools and a server designed to remotely administer the file systems of multiple Unix machines. For Mac OS X, there's also a graphical interface.

At its core, radmind operates as a tripwire. It is able to detect changes to any managed filesystem object, e.g. files, directories, links, etc. However, radmind goes further than just integrity checking: once a change is detected, radmind can optionally reverse the change.

Saving time is a daily concern for system administrators, especially with the increase of minor interventions requested by end-users: accidentally modified system preferences, third-party applications instability or misconfigured, wrong access rights, viruses, etc... Mac OS X is stable and robust but the capacity to (re)install completely a workstation or a server in a few minutes following a preset workflow that automatically restores the right disk image, configures and backs up what was planned by the administrator is a real benefit in terms of time and money. DeployStudio was designed this way, in order to manage flawlessly large deployments of workstations, servers and cluster nodes.

Flapjack is a scalable and distributed monitoring system. It natively talks the Nagios plugin format, and can easily be scaled from 1 server to 1000.

Flapjack tries to adhere to the following tenets:

it should be simple to set up, configure, and maintain
it should easily scale from a single host to multiple

munki is a set of tools that, used together with a webserver-based repository of packages and package metadata, can be used by OS X administrators to manage software installs (and in many cases removals) on OS X client machines.

munki can install software packaged in the Apple package format, and also supports Adobe CS3/CS4/CS5 Enterprise Deployment "packages", and drag-and-drop disk images as installer sources.

Additionally, munki can be configured to install Apple Software Updates, either from Apple's server, or yours