The Shadow File

Advanced defaults(1) Usage

Posted on

Want to change Finder’s preferred view to column view from the command line? There’s a defaults command for that. Want to alter trackpad behavior from a shell script? Yep, defaults can do it.

Lots of (most?) macOS and iOS settings manifest in the defaults subsystem (see Apple’s Preferences Programming Topics for Core Foundation and UserDefaults). What this means for you, the macOS user, is many of these settings exist somewhere on disk in the form of .plist files. You can find them in /Library/Preferences, ~/Library/Preferences, among other places.

There is a canonical utility provided by the operating system to access and manipulate many of these preferences: defaults(1).

The man page for defaults does a fair job of describing the command’s simpler uses, but you quickly hit more complex use cases that aren’t addressed. While there’s a fair bit of discussion on blogs and stackoverflow about the command’s use, most of that is incomplete as well. Many complain about defaults’s limitations, and recommend use of /usr/libexec/PlistBuddy. It’s true that defaults is limited in many ways, and PlistBuddy allows much more complex manipulations of plist files. But defaults works at a higher level of abstraction, using the defaults subsystem, without necessarily making assumptions about which specific file or files on disk represent the configuration. As such, if it’s possible to use defaults that’s preferable. We’ll dig into some of the more advanced uses, but before that here’s a quick plist refresher.

What is a plist?

It’s worth checking out the plist(5) man page, but the essence is that a plist is a serialized object graph. Unlike more generic object serialialization, plist composition is restricted to a handful of data types:

  • Top level object must be a dictionary or array
  • The top object may only be composed of:
    • Dictionaries
    • Arrays
    • Numbers (floats and integers)
    • Strings
    • Booleans
    • Dates
    • Data (an opaque blob of bytes)

As described above the only container types allowed at any level of the object graph are dictionary and array, which may themselves be composed of any of the above types.

Although property lists primarily exist in either binary or XML formats (there are also JSON and the Perl-ish NeXTSTEP formats), it’s easier to visualize the XML, so we’ll stick with that format. Here’s an example plist dictionary:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>top-level-key</key>
    <dict>
        <key>subkey1</key>
        <integer>1</integer>
    </dict>
</dict>
</plist>

Here we have a dictionary with a single key/value pair. The key being top-level-key, and the value being a nested dictionary. Strictly speaking a plist can be a dictionary or array, as described above, but as far as I know, defaults are always dictionaries. At any rate, defaults(1) can only modify dictionaries.

Technically, plist is just a data format. You may find files on your Mac that are plist data but are not user defaults/preferences. You can view them, but they’ll be nonsensical. In macOS and iOS, IOKit, CoreFoundation, and Foundation all have serialization APIs that write arbitrary object graphs out to XML or binary plist streams. Many applications save state in this form and then unserialize that state later.

Reading a plist

You can read preferences from a domain or a specific plist using the defaults command. I’ll describe that below, but its usefulness is limited. Defaults gives you output in the old perl-like NextSTEP format. This is human-readable but doesn’t convey the literal contents of the plist. Moreover, it doesn’t offer useful insight into what defaults will accept as input when writing values. Instead, use plutil(1). To view the XML version of a plist (whether it’s in binary format or already in XML), use plutil to convert it and write the output to standard out:

$ plutil -convert xml1 -o - ~/Library/Preferences/com.apple.finder.plist

In fact, this is a good alias to add to your shell:

$ alias plcat='plutil -convert xml1 -o -'
$ plcat ./foo.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>beast-mode</key>
    <true/>
    <key>top-level-key</key>
    <dict>
        <key>subkey1</key>
        <real>1</real>
    </dict>
</dict>
</plist>

Using defaults

Domains

User defaults are organized into domains, which are sort of an abstract and can take a few different forms. They’re often a reverse-DNS-style namespace that an application uses to store its settings. For example, Finder uses com.apple.finder. TextMate uses com.macromates.textmate. The reverse-DNS form isn’t strictly necessary, but it helps apps avoid namespace collisions with other apps.

There are a couple of other domain forms worth noting. First is the global domain, or NSGlobalDomain. Defaults accessed by an app that aren’t in that app’s own domain are looked up from the global domain.

There’s also the domain form of a literal file path. There are cases where, as far as I know, the only way to change a setting is to use this form (for example, many root-owned plists in /Library/Preferences).

A preference domain consists of a dictionary of keys and values, where keys are strings and the values are any of the plist types described above. You can view a list of domains available to your user:

$ defaults domains

You’ll get a long, comma separated list of domains containing user defaults.

Defaults Operations

You can use the defaults command to perform a variety of operations on a preference domain. The man page refers to these operations as “commands.” The operations you’ll use most include:

  • Reading default key
  • Adding a key
  • Changing an existing key’s value
  • Deleting a key

There are others as well (such as renaming a key) which are worth reading about in the man page.

So how do you manipulate settings with defaults? The basic form to read a default is:

$ defaults read <domain> <key>

To delete a key:

$ defaults delete <domain> <key>

To set or change a key, use the write command. We’ll focus primarily on this since writing defaults gets complicated quickly. This takes the form:

$ defaults write <domain> <key> <value>

The key from above is always a key in the top-level dictionary. There are ways to set values for keys in sub-dictionaries, which we’ll get into. It’s important to remember, though, that in all cases, you’re setting or modifying the value for a key in the outermost dictionary. When writing a key, if it doesn’t exist it will be created. In other words, modifying a key’s value is indistinguishable from adding a new key/value pair to the dictionary.

The value bit is where it gets tricky, and where a lot of the hidden flexibility of defaults comes in. We’ll explore this in depth.

Setting a default

Let’s say you want to write settings to domain foo.bar. Doing so would create (or modify) the file ~/Library/Preferences/foo.bar.plist. Alternatively, you could create this plist on your Desktop with ~/Desktop/foo.bar.plist. That gives us:

$ defaults write ~/Desktop/foo.bar.plist <key> <value>

If you wanted to set a key/value pair of beast-mode:True in foo.bar, you need to specify beast-mode as the key:

$ defaults write ~/Desktop/foo.bar.plist beast-mode <value>

Of course, you require a value. It’s not as simple as using “true” though.

$ defaults write ~/Desktop/foo.bar.plist beast-mode true
$ plcat ~/Desktop/foo.bar.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>beast-mode</key>
    <string>true</string>
</dict>
</plist>

Above, the ’true’ got interpreted as a string. Although in many cases, that might actually work, what you really intended was a boolean. So, ‘value’ gets a bit more complicated. Rather than true, in this case value must specify a type: -bool true

$ defaults write ~/Desktop/foo.bar.plist beast-mode -bool true
$ plcat ~/Desktop/foo.bar.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>beast-mode</key>
    <true/>
</dict>
</plist>

This time ’true’ got turned into a proper boolean. If you consult the man page, there are lots of type arguments, and some of them get pretty confusing:

  • -string (the default if nothing is specified)
  • -data
  • -int
  • -float (which translates, confusingly, into the <real> plist xml element)
  • -bool (case insensitive yes, no, true, or false)
  • -date
  • -array
  • -array-add
  • -dict
  • -dict-add

From the above list, string, int, float, and bool are pretty obvious. But for the rest, the man page gives no examples of their actual use. So you’re left scouring stackoverflow for examples, and then trying to work backwards to generalize rules for their use.

Here are some things that the most discussion misses. These allow us to describe a generalized use of defaults.

  1. You’re always modifying the value associated with a key in the top level dictionary
  2. Always specify the type even in cases where the type is implied
  3. You can provide the value in many forms, but the form that works in all situations (and is undocumented) is the XML plist fragment. When in doubt, use this.

The plist fragment form turns out to be pretty useful. It works like this; you could set a string value for the dictionary key, mystring, like so:

defaults write foo.bar mystring '<string>some-string-value</string>'

Note that when using the plist fragment form, the type is encoded in XML, so no type argument is required.

This is more verbose than simply:

defaults write foo.bar mystring -string some-string-value

Or even shorter:

defaults write foo.bar mystring some-string-value

Maybe you want to use the shorter forms when entering the command manually. But by generalizing the form, we can know a syntax that is guaranteed to work, and not wonder why a command should work but doesn’t.

If we’re interested in scripting defaults commands, or even using them in configuration management such as Ansible, it doesn’t really matter how verbose the command is.

Let’s go through each of the type arguments and explore how it’s used in our generalized form. I’ll also show the shorter form where appropriate

Type Arguments

String

Discussion: Plist values default to strings if you don’t specify.

Argument: -string

XML Plist Element: <string>

Examples:

defaults write foo.bar mystring '<string>some-string-value</string>'
defaults write foo.bar mystring -string some-string-value
defaults write foo.bar mystring some-string-value

Integer

Argument: -int

XML Plist Element: <integer>

Caveats: A float like 1.0 won’t be downcast to an integer when using the XML form. The command will fail. Be sure to use only actual integral values.

Examples:

defaults write foo.bar myint '<integer>1</integer>'
defaults write foo.bar myint -int 451

Float

Argument: -float

XML Plist Element: <real>

Example:

defaults write foo.bar myreal '<real>3.14</real>'
defaults write foo.bar myreal -float 3.14

Boolean

Argument: -bool

XML Plist Element: <true/>, <false/>

Examples:

defaults write foo.bar mystring -bool True
defaults write foo.bar mystring -bool YES
defaults write foo.bar mystring '<true/>'

Data

The “data” type is a sort of get-out-of-jail-free card. It’s all data, right? So “data” is opaque data that isn’t one of the other plist data types. All that matters is that’s it’s base64 encoded. Here are some examples of what it could be:

  • A jpeg file (look at <key>jpegphoto</key> in your own user record in /var/db/dslocal/nodes/Default/users)
  • Another plist (Look at <key>ShadowHashData</key> in your user record. It’s a base64-encoded binary plist that represents your hashed password.)
  • A serialized object that isn’t compatible with plist data types

Argument: -data

XML Plist Element: <data>

Caveat: Must be base64 encoded

Example:

defaults write foo.bar mydata "<data>$(echo -ne 'Quick to the point to the point no faking'|base64 -o -)</data>"

Date

Discussion:

This one’s a little confusing because the date must be formatted in a particular way. Namely, ISO-8601. And because you have more important things to worry about, that standard is:

Data elements and interchange formats – Information interchange – Representation of dates and times

Here’s the wikipedia article:

ISO 8601

Go read that and just before you decide that none of this is worth it because under 8601 there are still like a million date representations, come back.

I can’t find any formal documentation on how defaults wants its dates formatted. The man page says:

-date       Allows the user to specify a date as the value for the given preference key.

In my testing, it’s only happy with fairly narrow subset of ISO 8601. Specifically, it seems to want year + month + day + hour + minutes + seconds in the UTC time zone:

YYYY-MM-DDThh:mm:ssZ

Interestingly, you can give it a time with slightly nonsensical times, such as 62 minutes past the hour, and it will parse that time and write out a properly normalized time. For example:

2018-06-15T12:62:52Z

Becomes:

2018-06-15T13:02:52Z

Argument: -date

XML Plist Element: <date>

Caveats: input date must be a narrow subset of ISO 8601 (see discussion)

Examples:

    defaults write foo.bar mydate -date "2018-06-15T17:16:26Z"
    defaults write foo.bar mydate "<date>2018-06-15T17:16:26Z</date>"

Container Types

We’ve covered the primitive plist types and how to add them as values using defaults. Before moving on to the container types, let’s pause to discuss the ways they can get complicated.

For container types, there aren’t arguments I’m aware of to specify the data types they contain. For example, the man page gives as an example for adding an array:

defaults write somedomain preferenceKey -array element1 element2 element3

If you do that, you get:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>preferenceKey</key>
    <array>
        <string>element1</string>
        <string>element2</string>
        <string>element3</string>
    </array>
</dict>
</plist>

But what if the array is composed of integers? Or floats? Or a combination of types? Or what if you want one of the array members to be another array? Or a dictionary? I’m not aware of an equivalent of -int, -array, or -dict that can be applied to individual array members.

This is where the plist fragment trick gets really useful. It gives us a way to specify data types. So, instead of the above command which treats the array members as strings, we can do:

defaults write foo.bar myarray \
    '<array><integer>1</integer><real>3.14</real></array>'

So here are the container data types and how to add them as values.

Array:

Discussion: Adds an array of objects as the value for the given key. The objects may be arbitrarily complex.

Argument: -array

XML Plist Element: <array>

Caveats: There is no command line argument to specify the data types that make up the array. To do that, encode the array as XML.

Examples:

defaults write foo.bar myarray -array 1 2 # Adds 1 and 2 as strings
defaults write foo.bar myarray \
    <array><integer>1</integer><integer>2</integer></array>

Array-add:

Discussion: Allows adding of arbitrary objects of any plist type to the end of an array, creating the array if it doesn’t already exist. Added objects should be space separated even when in XML form (see examples). Added objects may be arbitrarily complex.

Argument: -array-add

Caveats: Members of an existing array cannot be deleted or rearranged. New items cannot be added anywhere but the end of an array. Arrays that require something more complex than items appended to the end should be added from scratch with -array.

Examples:

defaults write foo.bar myarray -array-add 1 2 3
defaults write foo.bar myarray -array-add \
    '<integer>1</integer>' \
    '<integer>2</integer>' \
    '<integer>3</integer>'
defaults write foo.bar myarray -array-add \
    '<dict><key>subkey</key><integer>7</integer></dict>'

Dict:

Discussion: Adds a dictionary as the value for the given key. The dictionary may contain arbitrarily complex objects.

Argument: -dict

XML Plist Element: <dict>

Caveats: There is no command line argument to specify the data types that make up the dictionary. To do that, encode the array as XML.

Examples:

#add a simple dictionary of key/string pairs
defaults write foo.bar mydict -dict key1 value1 key2 value2 

#more complex variants using XML representation of the dictionary
defaults write foo.bar mydict \
    '<dict> 
        <key>key1</key> 
        <real>3.14</real> 
    </dict>'

#nested dictionary
defaults write foo.bar mydict \
    '<dict> 
        <key>subdict</key> 
            <dict> 
                <key>subkey1</key> 
                <date>2018-06-15T17:16:26Z</date> 
            </dict> 
    </dict>'

Dict-add:

Discussion: Adds the provided key/value pair to a dictionary associated with the given key. The subdictionary is created if it doesn’t exist. If the subkey already exists in the subdictionary, its value is replaced. The value added to the subdictionary may be arbitrarily complex.

Argument: -dict-add

Caveats:

  • This only modifies a subdictionary at the second level of the object graph.
  • Subkey/value pairs cannot be deleted.
  • In order to perform these more complex operations, the entire subdictionary should be replaced using -dict-add.

Examples:

defaults write foo.bar mydict -dict-add key1 value1
defaults write foo.bar mydict -dict-add key2 '<real>3.14</real>'
defaults write foo.bar mydict -dict-add key3 \
    '<dict>
        <key>subkey1</key>
        <date>2018-06-15T17:16:26Z</date>
    </dict>'

Summary

That should be a fairly complete explanation of the various ways you can manipulate macOS/iOS defaults and their backing plists, with an emphasis on how to write keys and values to defaults dictionaries. Here’s a summary of key points:

  • Defaults are dictionaries of user preferences that are backed by plist files on disk
  • User defaults can be read and modified by /usr/bin/defaults
  • Reading and deleting can be performed per defaults domain, or per dictionary key
  • Writing a default always sets or modifies a specific key/value pair
  • Setting or changing a key/value pair is always an operation on a key in the outermost dictionary
    • This is true even when modifying a nested dictionary;
    • In effect, by adding to a nested container, you’re changing the value associated with a top level dictionary key
  • With primitive (int, float, etc.) plist types you can use either the type argument or encode the key’s value as an XML plist fragment
  • Using XML plist fragments you can add really complex structures to defaults dictionaries or subdictionaries, such as mutliple nested dictionaries and arrays
  • To do more complex operations on subdictionaries or arrays such as deleting elements, you need to re-add the entire structure as a value associated with the specified key using -array or -dict

In the next post I’ll share a tool I’ve built that will help automate the process of identifying where a user default is actually stored and then autogenerate a defaults incantation to write that setting.