How to parse and convert ini file into bash array variables?

Question

I'm trying to convert an ini file into bash array variables. The sample ini is as below:

[foobar] session=foo path=/some/path [barfoo] session=bar path=/some/path 

so these become:

session[foobar]=foo path[foobar]=/some/path session[barfoo]=bar 

and so on.

Right now, I could come up with only this command

awk -F'=' '{ if ($1 ~ /^\[/) section=$1; else if ($1 !~ /^$/) print $1 section "=" $2 }' 

Also, another problem is, it doesn't take spaces near = into consideration. I think sed is probably better suited for this job but I don't know how to hold and store a temporary variable for the section name in sed.

So any idea how to do this?

Answer 1

Gawk accepts regular expressions as field delimiters. The following eliminates spaces around the equal sign, but preserves them in the rest of the line. Quotes are added around the value so those spaces, if any, are preserved when the Bash assignment is performed. I'm assuming that the section names will be numeric variables, but if you're using Bash 4, it would be easy to adapt this to use associative arrays with the section names themselves as the indices.

awk -F ' *= *' '{ if ($1 ~ /^\[/) section=$1; else if ($1 !~ /^$/) print $1 section "=" "\"" $2 "\"" }' 

Note that you may want to also do the space removal that Khaled shows (on only $1 and section) since Bash variable names can't contain spaces.

Also, this method won't work if the values contain equal signs.

Another technique would be to use a Bash while read loop and perform the assignments as the file is read, using declare which is safe from most malicious content.

foobar=1 barfoo=2 # or you could increment an index variable each time a section is found while IFS='= ' read var val do if [[ $var == \[*] ]] then section=$var elif [[ $val ]] then declare "$var$section=$val" fi done < filename 

Again, associative arrays could fairly easily be supported.

Answer 2

I would use simple python script for this job since it has built in INI parser:

#!/usr/bin/env python import sys, ConfigParser config = ConfigParser.ConfigParser() config.readfp(sys.stdin) for sec in config.sections(): print "declare -A %s" % (sec) for key, val in config.items(sec): print '%s[%s]="%s"' % (sec, key, val) 

and then in bash:

#!/bin/bash # load the in.ini INI file to current BASH - quoted to preserve line breaks eval "$(cat in.ini | ./ini2arr.py)" # test it: echo ${barfoo[session]} 

Sure, there are shorter implementations in awk, but I think this is more readable and easier to maintain.

Answer 3

If you want to eliminate the extra spaces, you can use the built-in function gsub. For example, you can add:

gsub(/ /, "", $1); 

This will remove all spaces. If you want to remove spaces at the beginning or end of token, you can use

gsub(/^ /, "", $1); gsub(/ $/, "", $1); 

Answer 4

Here's a pure bash solution.

This is a new and improved version of what chilladx posted:

https://github.com/albfan/bash-ini-parser

For a really easy to follow initial example: After you download this, just copy the files bash-ini-parser, and scripts/file.ini to the same directory, then create a client test script using the example I've provided below to that same directory as well.

source ./bash-ini-parser cfg_parser "./file.ini" cfg_section_sec2 echo "var2=$var2" echo "var5[*]=${var5[*]}" echo "var5[1]=${var5[1]}" 

Here are some further improvements I made to the bash-ini-parser script...

If you want to be able to read ini files with Windows line endings as well as Unix, add this line to the cfg_parser function immediately following the one which reads the file:

ini=$(echo "$ini"|tr -d '\r') # remove carriage returns 

If you want to read files which have restrictive access permissions, add this optional function:

# Enable the cfg_parser to read "locked" files function sudo_cfg_parser { # Get the file argument file=$1 # If not "root", enable the "sudo" prefix sudoPrefix= if [[ $EUID -ne 0 ]]; then sudoPrefix=sudo; fi # Save the file permissions, then "unlock" the file saved_permissions=$($sudoPrefix stat -c %a $file) $sudoPrefix chmod 777 $file # Call the standard cfg_parser function cfg_parser $file # Restore the original permissions $sudoPrefix chmod $saved_permissions $file } 

Answer 5

Always assuming to have Python's ConfigParser around, one may build a shell helper function like this:

get_network_value() { cat <<EOF | python import ConfigParser config = ConfigParser.ConfigParser() config.read('network.ini') print (config.get('$IFACE','$param')) EOF } 

$IFACEand $param are the section respectively the parameter.

This helper then allows calls like:

address=`param=address get_network_value` || exit 1 netmask=`param=netmask get_network_value` || exit 1 gateway=`param=gateway get_network_value` || exit 1 

Hope this helps!

Answer 6

If you have Git available and are OK with the constraint of not being able to use underscores in the key names, you can use git config as a general-purpose INI parser / editor.

It will handle parsing out the key/value pair from around the = and discard insignificant whitespace, plus you get comments (both ; and #) and type coercion basically for free. I have included a complete working example for the OP's input .ini and desired output (Bash associative arrays), below.

However, given a config file like this

; mytool.ini [section1] inputdir = ~/some/dir enablesomefeature = true enablesomeotherfeature = yes greeting = Bonjour, Monde! [section2] anothersetting = 42 

…provided you just need a quick-and-dirty solution, and aren't married to the idea of having the settings in a Bash associative array, you could get away with as little as:

eval $(git config -f mytool.ini --list | tr . _) # or if 'eval' skeeves you out excessively source <(git config -f mytool.ini --list | tr . _) 

which creates environment variables named sectionname_variablename in the current environment. This, of course, only works if you can trust that none of your values will ever contain a period or whitespace (see below for a more robust solution).

Other simple examples

Fetching arbitrary values, using a shell function to save typing:

function myini() { git config -f mytool.ini; } 

An alias would be OK, here, too, but those are not normally expanded in a shell script [1], and anyway aliases are superseded by shell functions "for almost every purpose," [2], according to the Bash man page.

myini --list # result: # section1.inputdir=~/some/dir # section1.enablesomefeature=true # section1.enablesomeotherfeature=yes # section2.anothersetting=42 myini --get section1.inputdir # result: # ~/some/dir 

With the --type option, you can "canonicalize" specific settings as integers, booleans, or paths (automatically expanding ~):

myini --get --type=path section1.inputdir # value '~/some/dir' # result: # /home/myuser/some/dir myini --get --type=bool section1.enablesomeotherfeature # value 'yes' # result: # true 

Slightly more robust quick-and-dirty example

Make all variables in mytool.ini available as SECTIONNAME_VARIABLENAME in the current environment, preserving internal whitespace in key values:

source <( git config -f mytool.ini --list \ | sed 's/\([^.]*\)\.\(.*\)=\(.*\)/\U\1_\2\E="\3"/' ) 

What the sed expression is doing, in English, is

1. finding a bunch of non-period characters up to a period, remembering that as \1, then
2. finding a bunch of characters up to an equals sign, remembering that as \2, and
3. finding all the characters after the equals sign as \3
4. finally, in the replacement string
   • the section name + variable name is upper-cased, and
   • the value part is double-quoted, in case it contains characters that have special meaning to the shell if unquoted (like whitespace)

The \U and \E sequences in the replacement string (which upper-case that part of the replacement string) are GNU sed extension. On macOS and BSD, you'd just use multiple -e expressions to achieve the same effect.

Dealing with embedded quotes and whitespace in the section names (which git config allows) is left as an exercise for the reader. :)

Using section names as keys into a Bash associative array

Given:

; foo.ini [foobar] session=foo path=/some/path [barfoo] session=bar path=/some/path 

This will produce the result the OP is asking for, simply by rearranging some of the captures in the sed replacement expression, and will work fine without GNU sed:

source <( git config -f foo.ini --list \ | sed 's/\([^.]*\)\.\(.*\)=\(.*\)/declare -A \2["\1"]="\3"/' ) 

I predict there could be some challenges with quoting for a real-world .ini file, but it works for the provided example. Result:

declare -p {session,path} # result: # declare -A session=([barfoo]="bar" [foobar]="foo" ) # declare -A path=([barfoo]="/some/path" [foobar]="/some/path" ) 

Answer 7

With a lot of help from the answers in this post, as well as other on this site and some foogle (not a typo), I have come up with the following "solution" quoted because of the caveat at the end. I would prefer a pure bash method but everything I have seen either does not seem to work with my bash version or is rather cumbersome.

This fits the final category but still is less cumbersome than other options.

filename: functions

getSetting() { cat /dev/stdin | awk -v section="$1" -v key="$2" ' BEGIN { if (length(key) > 0) { params=2 } else if (length(section) > 0) { params=1 } else { params=0 } } match($0,/;/) { next } match($0,/#/) { next } match($0,/^\[(.+)\]$/){ current=substr($0, RSTART+1, RLENGTH-2) found=current==section if (params==0) { print current } } match($0,/(.+)=(.+)/) { if (found) { if (params==2 && key==substr($1, 0, length(key))) { print substr($0, length(key)+2) } if (params==1) { printf "%s\n",$1,$3 } } }' } 

filename: options.ini

[default] opt1=some value that matters not here opt2=another such value [section2] opt1=something different opt2=another something different 

filename: options This is the file that calls the function. so functions needs to be sourced here.

#!/bin/bash appPath="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" # Need to source the functions file to be able to use the above function source $appPath/functions defaultOpt1=$(cat $appPath/options/file.ini | getSetting 'default' 'opt1') defaultOpt2=$(cat $appPath/options/file.ini | getSetting 'default' 'opt2') sec2opt1=$(cat $appPath/options/file.ini | getSetting 'section2' 'opt1') sec2opt2=$(cat $appPath/options/file.ini | getSetting 'section2' 'opt2') 

And finally to access you need to source options:

filename: someapplication

#!/bin/bash app="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" # don't need to source functions again, but do need to source "options" file source $app/sources/options echo "$opt1 \n$opt2 \n$sec2opt1 \n$sec2opt2" 

I have looked at other options to "automatically" generate the variable by parsing the whole INI file but I am not sure there is anything really simple.

With this way, you can update the variables as you need to add them to your ini file. But, the caveat being it's not scalable in any meaningful way yet.

Answer 8

you can use a specialized ini parser like crudini
on Ubuntu install it with sudo apt install crudini

array.ini

[foobar] session=foo path=/some/path [barfoo] session=bar path=/some/path 

read it:

for section_name in $(crudini --get array.ini); do for var_name in $(crudini --get array.ini "$section_name"); do declare -A $var_name declare $var_name["$section_name"]="$(crudini --get array.ini "$section_name" $var_name)" done done 

here is the result:

$ for array_name in session path; do declare -p $array_name; done declare -A session=([barfoo]="bar" [foobar]="foo" ) declare -A path=([barfoo]="/some/path" [foobar]="/some/path" ) $