Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Perl Oneliners

Perl is a fantastic language that unfortunately has been eclipsed by other more popular languages.

Despite the lack of popularity for application programming Perl is still ubiquitous, you can use it everywhere for small tasks.

Even if you are not planning to use it to develop your next big application and even if you are not going to learn the depth of the language my hope that this book will provide you with useful oneliners that you can use as a helper tool for your daily tasks.

About the author:

Gabor has been programming since 1983 and he has been using Perl since 1994. Give or take a few months. He has been teaching Perl since 2000.

Install Perl

Follow the instructions on the Perl web site.

Linux

Most Linux distributions include perl, but if it is not installed then you can use your regular package-management system to install the perl or Perl package.

Debian/Ubuntu/etc.

apt install perl

macOS

Perl should come with the Operating System

Windows

Strawberry Perl is the community packaging of Perl for Windows.

Oneliners

Tutorial

Command line flags and parameters

For the official documentation visit perldoc perlrun or the same on MetaCPAN

  • -v print the version of perl
  • --version the same as -v
  • -V print defailed information about the perl. (e.g. compilation flags)
  • -e Execute the perl code that follows this flag.
  • -E Just like -e, but enable all extra features add to Perl since the 5.10 release.
  • -n Wrap the code provided using the -e or -E flags in a while loop to execute the code on every line of every file. The current line is in $_.
  • -p Do the same as -n and also print the possibly modified content of the current line stored in $_.
  • -i in-place editing (replace the processed file by the output)
  • -i.bak in-place editing with backup

Code snippets we use in the book

  • Regex (or regexp or Regular Expression) /bla/ or m/bla/ or m{bla} where bla can be any regular expression. See perlre or on MetaCPAN.
  • Substitution with regex s/bla/replacement/.
  • if - conditional
  • not - boolean negation
  • unless the same as if not
  • $. the current line number starting from 1

Best practices

  • Use version control (e.g. git) on all of your file, including your data files.
  • When doing in-place editing, if you don't have version control of the files you are editing then create a backup either before the process or during the process using -i.bak
  • Quotes: On Linux and macOS we usually use single-quotes around the perl code of the oneliner. On MS Windows AFAIK you cannot do that and thus there the outer quotes are double-quotes. This also means that the quotes, if used in the code, will need to be used differently. Therefore in general it is better to use q() instead of single quotes and qq() for double quotes.

Print the version of Perl

Before we get into the more interesting things, let's make sure that we have perl installed on the system and we can run it by printing the version number of the installed perl.

Open a terminal or on Windows open a CMD window and type in the following:

perl -v

The response I got on my Ubuntu Plucky Puffin (aka. 2025.04) is the following:

This is perl 5, version 40, subversion 1 (v5.40.1) built for
x86_64-linux-gnu-thread-multi
(with 48 registered patches, see perl -V for more detail)

Copyright 1987-2025, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at https://www.perl.org/, the Perl Home Page.

The long flag should give you the same output:

perl --version

Detailed version information

Using capital -V flag will give you a lot of details about the current version of perl. Most of which is probably not interesting so I won't bore you with explanations now. However, you are welcome to try in and marvell about the details.

perl -V

Latest Perl?

Perl.org indicates that the latest stable version is 5.42. It is recommended that you use the latest stable version, but for the examples in this book you can probably use much older versions of Perl as well.

Print a number, use Perl as your calculator

You can execute any perl code that you provide on the command line as the value of the -e or the -E flag. The difference between the two is that -E enables all optional features and builtin functions that were added since version 5.10 came out in, well a very long time ago.

The first such extra that we get using -E is the say function.

  • print will print the value after it
  • say will also print the value after it, but it will also print a newline. \n at the end.
perl -e "print 42"


perl -E "say 42"

Forgetting the -E flag

What happens if you try to use the say function, but mistakenly use the -e flag? You get an error:

$ perl -e "say 42"
Number found where operator expected (Do you need to predeclare "say"?) at -e line 1, near "say 42"
syntax error at -e line 1, near "say 42"
Execution of -e aborted due to compilation errors.

Calculator

You can use perl on the command line as a calculator:

$ perl -E "say 19 * 23"
437

$ perl -E "say 19 + 23"
42
  • -e
  • -E
  • print
  • say

Print a string

Strings in perl are either inside single-quotes ('') or double-quotes ("").

perl -E "say 'hello world'"

As in most other programming languages if you would like to use a quote character inside a string you will need to escape it by putting a backslash \ in front of it.

perl -e "print 'hello \'nice\' world'"

This is doubly true for oneliners as the whole perl expression is also inside some kind of quotes.

$ perl -E "say 'hello \"nice\" world'"
hello "nice" world

However, perl allows you to use q() instead of single-quote and qq() instead of double quotes. That means we can rewrite the earlier example using either of them

perl -E "say q(hello 'nice' world)"
hello 'nice' world

$ perl -E "say qq(hello 'nice' world)"
hello 'nice' world

The difference betweem single- and doble-quotes is that in double-quotes variables are interpolated. We'll see that in later examples. In our current examples that did not matter.

Using grep on Windows as well

How can you search for all the occurrences of the 'secret' in your xml files?

On Linux and macOS you would probably use grep or egrep. You can also install some kind of grep on MS Windows as well, but if you already have Perl installed then you can use it as well.

In a single file

perl -ne "print if /secret/" main.xml
  • -e tells perl to use the parameter as perl code
  • -n tells perl to execute that code on every line of the input file and on each iteration put the current line in the default variable called $_.
  • print without parameters will print the content of $_.
  • /secret/ is regex (the slashes are the delimiters). If no =~ is provided then it works in the content of the default variable called $_.

The above line is the same as the one below:

perl -n -e "print $_ if $_ =~ /secret/" main.xml

Use Deparse to convert the oneliner into full script to make it more readeble.

$ perl -MO=Deparse -ne "print if /secret/" main.xml

LINE: while (defined($_ = readline ARGV)) {
    print $_ if /secret/;
}
-e syntax OK

In multiple files

As Windows does not handle wildcards on the command line, we cannot supply *.xml and expect it to handle all the xml files. We help it with a small BEGIN block. $ARGV holds the name of the current file

perl -ne "BEGIN{ @ARGV = glob '*.xml'} print qq{$ARGV:$_} if /secret/"
  • The BEGIN {} construct will be executed once, before we start looping over the lines
  • glob '*.xml' is an expression in perl returning the list of all the files in the current folder with xml extension.
  • @ARGV is the global variable that holds the command line parameters. Usually it is filled by perl when a script starts. Here we fill it ourselves from inside the perl code.
  • The -n flag tells perl to iterate over every line in every file listed in @ARGV. On every iteration the name of the current file is in the $ARGV variable and the current line is in the $_ variable.
  • In this example we use the reversed if-statement (do something if some condition).
  • /secret/ is a regex checking if the current content of $_ matches the series of characters: secret.

We can use the B::Deparse library to ask perl to show how it understands our code:

$ perl  -MO=Deparse -ne "BEGIN{ @ARGV = glob '*.xml'} print qq{$ARGV:$_} if /secret/"
LINE: while (defined($_ = readline ARGV)) {
    sub BEGIN {
        use File::Glob ();
        @ARGV = glob('*.xml');
    }
    print ':main.xml' if /secret/;
}
-e syntax OK

Rename many files

  • Rename every .log file adding an extra extension of .old
perl -e "rename $_, $_ . '.old' for glob '*.log'"
  • Rename every .log file replacing .log with a timestamp
perl -e "$t = time; rename $_, substr($_, 0, -3) . $t for glob '*.log'"
  • rename
  • glob

Replace a string in many files

You have a bunch of text files in your directory mentioning the name:

"Microsoft Word"

You are told to replace that by

"OpenOffice Write"

perl -i -p -e "s/Microsoft Word/OpenOffice Write/g" *.txt

-i = inplace editing
-p = loop over lines and print each line (after processing)
-e = command line script
  • -i
  • -p

Replace one line with an empty row

perl -i -p -e 's/^date://' *.txt

Remove one line from a file

  • TODO: use version control or -i.bak
  • TODO: mention that structured files such as YAML migh be better edited with a dedicated parser. What if the value associated with the key isn't on the same line (as is common with arrays and hashes)?

I had a thousand YAML files that, for historical reasons, had a field called date:. At one point update the code processing these files and wanted to remove this date field from all the files.

For example the one that starts with date:

perl -i -n -e 'print $_ if $_ !~ /^date:/' *.yaml
  • -i means inplace editing, that is the output replaces the input file
  • -n means go over the input file(s) line by line.
  • -e means, here comes the code.

In the code

  • $_ represents the current line
  • /^date:/ is a regular expression where the two slashes are the delimeters. ^ means our regex has to match the beginning og the string. date: is just the string we match.
  • !~ is the regex not match operator. (=~ is the regex match operator)
  • So this expression $_ !~ /^date:/ checks if the current line starts with date:.
  • Just like natural languages, perl also allows reversing the order of a conditional statement. So instead of if (condition) { do_something } we can write do_something if condition.
  • We print the current line if id does not start with date:.

We can improve our code:

$_ is the default for print

We don't need to explicitelly print the content of $_ if we just write print without telling perl what to print it will print the content of $_.

perl -i -n -e 'print if $_ !~ /^date:/' *.yaml

Use boolean not and the matching operator

Intead of the !~ "not match" operator we could use the match operator =~ and then use a boolean not:

perl -i -n -e 'print if not $_ =~ /^date:/' *.yaml

This in itself probbaly isn't an improvement, but the matching operator also defaults to work on $_ if no parameter provided. Actually for this to work we even remove the match operator. If we have a regular expression without a matchin (or not-matching) operator then it defaults to work on the content of $_. Neat.

perl -i -n -e 'print if not /^date:/' *.yaml

Use unless

Another potential improvement to our code is using the unless keyword which is the same as if not.

perl -i -n -e 'print unless /^date:/' *.yaml

Remove comments from many files

In older versions of Rust when we created a new crate using cargo new it added a comment to the generated Cargo.toml file:

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

On one hand this was a good idea as it helped people to find out more about the various keys one can use in the Cargo.toml file, on the other hand it is annoying as it makes the file wide. I know, I can remove it.

And this is exactly what I deceided to do. Remove it from all the files in all the crates. Luckily, in this case, all the crates are located in the same folder:

perl -n -i -e 'print unless /^#/' */Cargo.toml
  • Remember to use -i.bak if you don't have the files in version control. Better yet, start using version control before you use this one-liner.
  • -n tells perl to execute the command on every line.
  • print unless /^#/ means: "Print the current line unless it starts with a # character." (Both print and the // regex match apply to the (invisible) $_ variable that in this situation contains the current line.

Change encoding

Convert all the .srt files that are Windows Hebrew encoded to UTF-8 keeping a backup copy of the original file with a .bak extension.

-i    - inplace editing
.bak  - generate backup with that extension
-M    - load  module as 'use' would do
-p    - go over line by line on the file, put the line in $_, execute the
        command on it and then print the result.
        In case of inplace editing, save it back to the file.

perl -i.bak -MEncode -p -e "Encode::from_to($_, 'Windows-1255', 'utf-8')" video.srt

use Encode;

foreach $file (@ARGV) {
    copy $file, "$file.bak";
}
while (<>) {
   Encode::from_to($_, 'Windows-1255', 'utf-8');
   print $_;
}
  • -i
  • -M
  • -p
  • Encode

print the 3rd field of every line in a CSV file

You have a number of csv files, you want to print the 3rd field of each row of each file.

perl -n -a -F, -e 'print "$F[2]\n"' *.csv

-n  = loop over lines but do NOT print them
-a  = autosplit by ' '
-F, = replace the split string by ','

You want to make sure all the rows are 4 elements long. Print out file name and line number of all the bad rows.

perl -a -F, -n -e 'print "$ARGV:$.\n" if @F != 4' data.csv
  • CSV
  • -n
  • -a
  • -F

Print all usernames from /etc/passwd

perl -n -a -F: -e 'print "$F[0]\n"' /etc/passwd

Double-space a file

Given a file

One
Two
Three

Convert it to

One

Two

Three


perl -i.bak -n -E 'say' src/examples/lines.txt

Don't add the last empty row

$ perl  -n -E 'print "\n" if $. > 1; print' src/examples/lines.txt
one

two

three

one
two
three

Add row numbers to file

$ perl  -n -E 'print "$.) $_"' src/examples/lines.txt
1) one
2) two
3) three

Use module to sum numbers

The List::Util module provides a function called sum that can add together numbers. We can use it in our oneliner as well.

$ perl -E 'use List::Util qw(sum); say sum(2, 3, 17)'
22

Sum numbers in a file

1
3
7
10

$ perl -n -E 'END { say $c } $c += $_' src/examples/numbers.txt
21

Use module to add numbers in a file

1 2 13

$ perl -n -E 'use List::Util qw(sum); say sum(split)' src/examples/numbers_in_line.txt
16

Check if a number is prime

Originally Written by Abigail in 1998 it listed in the JAPH file.

perl -E 'say "Prime" if (1 x shift) !~ /^1?$|^(11+?)\1+$/' 23

Remove leading spaces

perl -pE 's/^ +//' src/examples/text_with_spaces.txt

perl -pE 's/^[ \t] +//' src/examples/text_with_spaces.txt

Using \s looks easy but this will also eliminate every row that has only spaces and/or tabs in it.

perl -pE 's/^\s +//' src/examples/text_with_spaces.txt

Remove trailing spaces

$ perl -pE 's/[ \t]+$//' src/examples/text_with_spaces.txt

This is not good as it will convert all the file into one long line:

perl -pE 's/\s+$//' src/examples/text_with_spaces.txt

This fixes it:

perl -nE 's/\s+$//; say' src/examples/text_with_spaces.txt

perl -lpE 's/\s+$//' src/examples/text_with_spaces.txt

Insert rows in many TOML files

My Rust Maven web site contains several books written in Markdown and I am using mdbooks to convert them to web sites. Each such book as a configuration file called book.toml.

In the configuration file one can add an entry that looks like this:

[rust]
edition = "2024"

By default this entry is not in the files, but at one point I wanted to add it to all the files.

In the generic case it might be a better idea to use a real TOML-handling library to make these changes, but as I know all of my book.toml files look the same I could safely use a regex-based solution.

I am using a Linux system, but if you are using git-bash from git-scm on Windows, you can probably do the same.

Example book.toml file

This is how such a file looks like:

[book]
authors = ["Gabor Szabo"]
language = "en"
src = "src"
title = "Rust Programming"

# Add github icon to the menu
[output.html]
git-repository-url = "https://github.com/szabgab/rust.code-maven.com"
git-repository-icon = "fa-github"
edit-url-template = "https://github.com/szabgab/rust.code-maven.com/edit/main/books/rust-programming/{path}"

[output.markdown]

# Handle Youtube video embedding and the footer
[preprocessor.embedify]
footer.enable = true
footer.message = "Copyright © 2025 • Created with ❤️  by [Gábor Szabó](https://szabgab.com/)"

Insert rows

We want to insert 3 rows in every file (two rows with content and an empty row for readability) just above a line containing a specific content.

perl -p -i -E 'say qq{[rust]\nedition = "2024"\n} if /# Add github/' */book.toml

  • -E tells perl to execute the command that follows it.

  • -p tells perl to iterate over the lines of every file following the command and to execute the command for every line while the content of the current line is in the $_ variable. after executing the command print the content of $_ that might have been modified.

  • -i meant inline editing. That is, instead of printing the content of $_ to the screen write it back to the original file.

  • -i.bak Would tell perl to create a backup of the file before changing it. I don't use it as my files are in version control so I can easily return to the previous version and I don't want to have lots of .bak files laying around.

In our case we don't modify the lines. Instead we say qq{[rust]\nedition = "2024"\n} will print the 3 lines when the current line (in the invisible $_ variable) match the regex /# Add github/. The slashes are the delimiters of the regex.

Change a value in many TOML files

My Rust Maven web site contains lots of small Rust examples. Each one is a crate (a library) and thus each one has a Cargo.toml file that has the edition field. Some are older examples where the edition is 2021, others are newer with edition being 2024. First I wanted to see this information, which file has which editions, and then I wanted to update every example to be in the newer edition.

I am using a Linux system, but if you are using git-bash from git-scm on Windows, you can probably do the same.

The find . -name Cargo.toml command lists all the Cargo.toml files in the directory tree starting the current directory represented by dot. There are a lot.

I can count them if I pipe the result through the wc command with the -l flag.

$ find . -name Cargo.toml | wc -l
1342

If I pipe the results through xargs then I can execute a command on the resulting list of files, for exampl I can use grep edition to extract all the lines that contain the word edition:

$ find . -name Cargo.toml | xargs grep edition
...

I can take the result of that command and pipe through wc again to get the number of lines:

$ find . -name Cargo.toml | xargs grep edition | wc -l
1342

So the number of edition lines is the same as the number of file.

I can use the grep 2024 to see filter all the edition-lines that contain the value 2024

$ find . -name Cargo.toml | xargs grep edition  | grep 2024

... and I can count those:

$ find . -name Cargo.toml | xargs grep edition  | grep 2024 | wc -l
534

I can also use the -v flag of grep to filter the lines that do NOT contain 2024, these are the ones I will want to edit.

$ find . -name Cargo.toml | xargs grep edition  | grep -v 2024

... and I can count those as well:

$ find . -name Cargo.toml | xargs grep edition  | grep -v 2024 | wc
808

Looks good. 808 + 534 is really 1342

I did not check if all the non-2024 editions indeed contain 2021, but it does not really matter as I would find that out run the above commands again after I make the changes.

Change the edition

In order to make the change I use a little expression in perl and pipe through all the file.

  • -i tells perl to replace each file with the results. If I did not have all the files in version control I'd use -i.bak to create a backup before making the changes, but I have all these examples in git.
  • -p tells perl to go over each line in the files and print them.
  • -e tells perl to execute the statement following it. As we have -p, perl will execute this statement on every line.
  • s/edition = "2021"/edition = "2024"/ is a rather simple regular expression substitution. Whatever is matched by the regex between the first two slashes is replace with the string between the 2nd and 3rd slash.
$ find . -name Cargo.toml | xargs perl -i -p -e 's/edition = "2021"/edition = "2024"/'

So that's it I change all the files.

Verification

Now lets do some verification.

As all of my examples are in version control I can run git diff to see the changes. I did that. Thet looked good.

I can also run git status to see which files have changed and I can pipe it through wc -l to see how many files changed:

$ git status | wc -l
    813

Oh. That baffled me. How is that I only had 808 files that were not 2024, but 813 have changed?

That's actaully easy, the git status command prints some text above and some below the list of files and wc countd those lines as well.

So let's use the --porcelain flag. That will only print the filenames without any extra:

$ git status --porcelain | wc -l 805

Oups. Now it say only 805 changed and we had 808 lines to change.

Troubleshooting

I ran the counters again and they told me all the 1342 edition lines have now 2024. So I did not know what's going on.

The only thing I could think of is that there might be some Cargo.toml files that have the edition line more than once.

So I set out to try to count the number of edition lines in each file

Count how many time edition appears in each Cargo.toml file

For example I used this command:

$ find . -name Cargo.toml | xargs grep edition | grep -v 2024 | cut -d: -f1 | sort | uniq -c | head

and this command

$ find . -name Cargo.toml | xargs  perl -MData::Dumper -n -E '$c{$ARGV}++ if /edition/; END {say Dumper \%c}'

Nothing.

Then it occured to me:

What if I have some Cargo.toml files in the tree that are not under version control? Those would still be counted when I was counting the edition lines, but they would not appear in the git status output as they are not under version control.

So I ran this command to see if there might be any files that are not in any of the /src/ folders.

$ find . -name Cargo.toml | xargs grep edition | grep -v 2024 | grep -v '/src/'

Indeed it showed me 3 files. 3 html files that were generated by mdbook. There were only 3 because most of the books did not have the generated version on my disk and the book (the rust async book only had 3 examples with older editions.

Huh. I spent quite a few minutes feeling totally lost, but I am glad I finally found the source of that difference.

With that I could commit the changes.

Resources

Some other explanations about oneliners