Zero Padding Brace Expansions in the Korn Shell

Both bash and zsh shells support leading zeros in ranges:

$ echo {1..10}
1 2 3 4 5 6 7 8 9 10
$ echo {01..10}
01 02 03 04 05 06 07 08 09 10
$ echo {001..010}
001 002 003 004 005 006 007 008 009 010

From the bash manpage section on brace expansion:

Supplied integers may be prefixed with 0 to force each term to have the same 
width. When either x or y begins with a zero, the shell attempts to force all 
generated terms to contain the same number of digits, zero-padding where necessary

If you want a comma separated range of numbers, you can do something like:

$ echo {01..10} | tr ' ' ','

The Korn shell does not support the leading zero syntax in brace expansion. However it does support an equally powerful alternative syntax using %0d:

$ echo {1..10}
1 2 3 4 5 6 7 8 9 10
$ echo {01..10%02d}
01 02 03 04 05 06 07 08 09 10
$ echo {01..10%03d}
001 002 003 004 005 006 007 008 009 010

$ echo {01..10%02d} | tr ' ' ','  

Finally, you can also use this style of syntax to create a range of leading zero hexadecimals:

$ echo {7..18%02x}  
07 08 09 0a 0b 0c 0d 0e 0f 10 11 12

Intel Processor Power Management

Modern Intel processor power management is based on a combination of two techniques: changing the CPU frequency (P-states) and using idling power states (C-states). A third technique, used on older processors but rarely on modern processors, is throttling of the CPU (T-states).

P-states are voltage-frequency pairs that set the speed and power consumption of an Intel processor. When the operating voltage of the processor is lower, so is the power consumption.

C-states are idle power saving states, in contrast to P-states, which are execution power saving states. During a P-state, the processor is still executing instructions, whereas during a C-state (other than C0), the processor is idle, meaning that nothing is executing. Thus C-states are idle states and P-states are operational states. C-states and P-states are orthogonal, i.e. one can vary without affecting the other. C-states have been available for processors targeted at the mobile market for many years but have been only available for the server processors since Nehalem.

In the C0 state, a processor executes instructions. A processor running in any other C-state is somewhat idle. The higher the C-state number, the deeper the CPU sleep mode and the more power saved. The downside to C-States is the latency when reverting to C0. Some C-States have submodes with different power saving latency levels. C1 is always available but which other C-states and submodes are supported is processor-specific.

If you want the absolute minimum latency, set the kernel parameter idle to poll. This keeps the processor(s) at C0 even when idle. Note that keeping processors at C0 can interfere with hyperthreading, i.e. logical cores.

From the the current Linux Kernel Parameters document:

idle=		[X86]
			Format: idle=poll, idle=halt, idle=nomwait
			Poll forces a polling idle loop that can slightly
			improve the performance of waking up a idle CPU, but
			will use a lot of power and make the system run hot.
			Not recommended.
			idle=halt: Halt is forced to be used for CPU idle.
			In such case C2/C3 won't be used again.
			idle=nomwait: Disable mwait for CPU C-states

The higher the P-state, the lower the frequency and voltage at which a processor runs. Once again, the number of P-states is again processor-specific and implementations differ across the various processor families. However, P0 is always the highest-performance state. Higher P-state numbers represent slower processor speeds and lower power consumption. By the way, P-state technology is known as SpeedStep in Intel processors.

Linux supports a CPUfreq subsystem which can be used to modify P-States. If it is enabled on a particular Linux distribution, you will find the relevant files under /sys/devices/system/cpu/. There will be a separate subdirectory for each processor with a cpufreq subdirectory in each processor directory, holding a number of files that define the parameters for CPUfreq.

Finally, a useful tool is cpufreq-info which can be used to retrieve cpufreq-related kernel information.

On-disk File Timestamps

The Single Unix Specification, Base Definitions (XBD), Section 4.8 entitled “File Times Update” states

“An implementation may update timestamps that are marked for update immediately, or it may update such timestamps periodically.”

This means that, for example, that file read and write operations are free to set the appropriate flags in the in-memory structures and do the actual updating of the on-disk filesystem structures at a later time.

Assuming periodically means from time to time, it implies that a POSIX-compliant operating system is free to update it’s on-disk structures when it is convenient for the operating system to do so. This means that, should the operating system crash, the on-disk recorded times might not reflect the actual true last access times.

BSD and Solaris UFS are two filesystems that exhibit such behavior. The Solaris UFS file systems does not update access times unless the disk is accessed or a certain events occur. BSD only periodically updates it’s on-disk filesystem timestamps. On BSD, all

timestamps that are marked for update are updated when the file ceases to be open by any process or before a fstat(), fstatat(), fsync(), futimens(), lstat(), stat(), utime(), utimensat(), or utimes() is successfully performed on the file.

From a forensics viewpoint, this implies that you need to understand the underlying filesystem on-disk timestamp updating mechanism before you can reliably rely on an individual file timestamp.

IPv4 Link Local Addresses in Fedora

A network link-local address is an address that is valid only for use within the network segment or the broadcast domain that a host is connected to. Link local addresses can be layer 2 or layer 3. For example, a 48-bit Ethernet MAC address is one form of a layer 2 link-local address. Link-local addresses are generally not guaranteed to be unique beyond a network segment. Because of this, layer 3 switches (routers) do not forward packets with layer 3 link-local addresses. For Layer 3 IPv4 link-local (IPv4LL) addresses, RFC3927 specifies the address block

Zero-configuration Networking (zeroconf) is a set of technologies that automatically creates a usable TCP/IP network when hosts or network peripherals are interconnected. It does not require system administrator intervention or configuration servers. It consists of three core technologies: assignment of numeric network addresses for networked devices, automatic distribution and resolution of computer hostnames, and automatic location of network services, such as printers. Without zeroconf, a system administrator must set up Dynamic Host Configuration Protocol (DHCP) and Domain Name System (DNS), or configure each host’s network settings manually. Microsoft uses the term APIPA (Automatic Private IP Addressing) instead of zeroconf in their operating systems. Zeroconf is sometimes called AutoIP.

Avahi is a zeroconf implementation for Linux and BSD. It implements IPv4LL, mDNS and DNS-SD. It is installed by default on Fedora and a number of other Linux distributions. Most development work was done in the period 2004 to 2011 and it is now regarded a mature technology. The pseudo-daemon avahi-autoipd implements the IP4LL functionality. It can be used as stand-alone address allocator or as plugin for a DHCP client such as dhclient where it would be used as the fallback IPv4 address alocator if no DHCP server is found.

Avahi has been installed by default on Fedora for a number of years. For example, on Fedora 17:

# ps -ef | grep avahi
avahi      267     1  0 Aug08 ?        00:00:00 avahi-daemon: running [fedora.local]
avahi      307   267  0 Aug08 ?        00:00:00 avahi-daemon: chroot helper

# avahi-autoipd -h
avahi-autoipd [options] INTERFACE
    -h --help           Show this help
    -D --daemonize      Daemonize after startup
    -s --syslog         Write log messages to syslog(3) instead of STDERR
    -k --kill           Kill a running daemon
    -r --refresh        Request a running daemon refresh its IP address
    -c --check          Return 0 if a daemon is already running
    -V --version        Show version
    -S --start=ADDRESS  Start with this address from the IPv4LL range
    -t --script=script  Action script to run (defaults to
    -w --wait           Wait until an address has been acquired before
       --force-bind     Assign an IPv4LL address even if a routable address
                        is already assigned
       --no-drop-root   Don't drop privileges
       --no-chroot      Don't chroot()
       --no-proc-title  Don't modify process title
       --debug          Increase verbosity
# avahi-autoipd -V avahi-autoipd 0.6.31 # cd /etc/avahi # ls -l total 20 -rwxr-xr-x. 1 root root 2509 Aug 9 2012 avahi-autoipd.action -rw-r--r--. 1 root root 1749 Aug 9 2012 avahi-daemon.conf drwxr-xr-x. 2 root root 4096 Aug 9 2012 etc -rw-r--r--. 1 root root 1121 Aug 9 2012 hosts drwxr-xr-x. 2 root root 4096 Aug 19 2013 services

If you do not need IPv4LL functionality, consider disabling Avahi:

# systemctl disable avahi-daemon.service 

PAE – Physical Address Extension

PAE (sometimes called PAE-X86) was first implemented by Intel in 1995 in certain models of the Pentium Pro. The PAE IA-32 architecture supports 4 additional address lines for memory selection, so physical address size increases from 32 bits to 36 bits. This enables 4Gb x 2*4 = 64Gb of physical memory to be accessed. The CPUID flag for PAE support is, naturally, PAE. Since then virtually every 32-bit CPU produced comes with PAE support.

Even with a PAE-enabled CPU, the IPL32 programming model (IA32 flat memory mode) continues to be able to only see 4 Gb of memory. You need a PAE-aware Linux kernel which uses an 3-level page table hierarchy to access more than 4Gb of memory. Depending on how they were written, applications that wish to access more than 4Gb of memory may need to be modified.

The Linux kernel first supported PAE as a build option in version 2.3.23. Initially seperate 32-bit and 32-bit PAE kernels were the norm for Linux distributions. Nowadays, however, major Linux distributions, except for Debian, only provide a 32-bit PAE-enabled kernel.

For a detailed explanation of how PAE page table work, I recommend this Wikipedia article and this Microsoft article.

Note that 64-bit kernels do not need or use PAE.