[Music]
okay our final speaker today is tan and
he's going to be talking about
configuration management
good afternoon and good day to everyone
who's still sticking around with me in
this linux conference australia welcome
to my playtime with configuration how
did i get from shell script to ansible
galaxy
before i begin
i would like to acknowledge the
traditional custodian of the land which
the talk is being streamed and record i
would also like to pay my respect to
their elders past and present
so what is this about this is actually a
summary of my journey on how i discover
the tools that i use to do my
performance
to performing my configuration
management
i started doing configuration on linux
system back in year 2000
it was not fun i actually started doing
it manually
and then i i upskill myself to start
using shell script
but it's not fun especially when i have
more than one distro to manage in my
fleet it become very complex and
complicated
to to to manage the shell script
and later i discovered configuration
tools the power of cf engine and puppet
but for whatever reason i never liked
either one of them
and eventually in 2012 i actually
discovered ansible
when i first started to use siber i only
used the very basic module like scrib
to run my shell script
and over the time i slowly understand
the power of ansible module and i move
my shell script into ansible module
and eventually i
organize my ansible into rows
and today i'm going to share the two
extreme ends of my journey
the dark age when i use salscript for
configuration and the bright galaxy when
i start to play with nc bird
but
who am i
my name is dan sir heng
that's the cartoon sketch of me
but in australia i'm actually called sir
hang ten
also known by my last name tan
but some people still calling me by my
first name which is sir
uh i answer to anyone of that
and um i'm a linux user a linux
enthusiast
which i have been
playing or learn about computers since
the 90s i start with playing computer
games
not too sure anyone know karateka
command conquer counter-strike and
counter-strike global offense and so on
those are a few games i play in previous
and then
when i graduated in
year 2000 i got employed as a
junior system administrator
web developers kind of position in an
e-commerce
company
which they have a fleet of
five physical servers that's running on
linux
and they're providing the e-commerce
solution to the customer and that's when
i discover and learn about linux
and since then i've been learning living
and learning linux
linux become my bread and butter so much
that i'm now being employed as linux
system administrator
if i'm not wearing my geeky hat i
actually love watching anime cartoon and
all sorts of movies and i
if i have time i tend to build my lego
if i have place to actually display it
if not there will be n up in the box
behind me it's still in the box
so
what is this configuration and why do i
think it's important especially to linux
system administration like myself
configuration is very very important
because they actually make the it system
work
imagine that you have a apache server
that you installed
but you didn't do any setup and
configuration
do you think you'll be able to
serve your company website
how about you have just installed
mariadb
do you think you'll be able to log in
and enter data without proper setup and
configuration i doubt you can
and that's why
we need
a discipline an engineering process
configuration management that will help
us to establish and maintain the
consistency in the it infrastructure by
ensuring our design
our system
the state of our application and
services in the environment they are
running in being managed tracked and
trusted
to me
a tool that do that is very important
because it will
be the utility to ensure the consistency
that we want in our infrastructure it
will also be the enablement to ensure
that we can scale up or down our
infrastructure and software
and lastly the most important bit to me
is he actually remove and reduce all the
repetitive and boring bits of system
administration and configuration tasks
that i've been doing manually
now
let me bring you back in time to the
dark edge when i do shell script
configuration
i will need the
the menu apache web server installation
on center 7 and then i'm going to
customize my web server landing page
so
first i need to do
is
sudo to root
and then i'm going to install
the web server with a php module
after installation of apache web server
complete we need to enable it and start
the service and the next thing i'm going
to do is i'm going to create a
customized landing page as well as a php
info page
let's check the
customize web server
yup and let's see the
info page
for the php and we would be using
php version 5. installing apache web
server with a customized index page is
very simple and straightforward with
manual steps
but that is only possible or feasible if
you have one two
three
maybe
five web servers
but in a web server farm usually we have
more than 10
hundred or thousand web servers is going
to be a lot of menu typing when we do it
manually
therefore i started to use shell script
i created a few cell script based on the
manual step that i have done for example
centos 7 setup.sh i have jump install
and the customize
index page but
this file is only cater for
centos 7 installation it will not work
for other distro for example ubuntu
because ubuntu used a different
application installation method called
apt and the apache web server in ubuntu
is not http is apache too
and therefore i need to create a
separate script for it
uh if i have other digital i might need
to create more script to educator for
that those web server installation
i created one third script which
combines the center 7 and ubuntu into
this script which this script actually
have a detection of operating system
that the script is going to be run on
and then based on the operating system
on this show for example in this case
centos
or ubuntu it will install httpd or
apache web server
now i'm going to run this guy
against
uh
my web server which is in this file
7 2 and ubuntu and i'm going to run it
in a
in a for loop
the apache web server setup is complete
on that 3
system let us go into them and verify
the landing page which is the customized
landing page that i have
same go to here and
the ubuntu
as you can see from the demonstration
using shell script as your configuration
tools is going to be disaster
imagine that you're trying to set up 500
servers used completely by shell script
and access search you take a while and
you will not be
nice especially sales script is not a
configuration tools to start with
and that's why in 2012 i start looking
for a better tools to replace my shell
script configuration
i have used cf engine in the past and
just started to learn puppet during that
time but neither of these tools fulfill
what i think
i need in a configuration
management tools
so
what i want is something too simple to
start
which can quickly use and
adapt my shell script instead of
rewriting a whole bunch of configuration
uh into the tools
i
do not want to have extra connection
maybe just use ssh like how i have run
my shell script over the ssh
channel i definitely do not want to have
additional package to be installed on
the target notes that i'm going to
manage
because installing additional package
there mean that i need to do extra work
to ensure
the application or the daemon that
running on the target notes is compliant
and
workable in the
with the masternodes or the management
nodes
i want
a configuration tool that
can use
an easy and understandable syntax
something that descriptive
uh telling us what is going to do what
is going to run
and lastly i want the configuration
setup to be item potent
which means that configuration can be
re-applied as many times as it runs
without changing the end result beyond
what have been defined
and that's when
i discovered ansible
ansible was first introduced by michael
diehan in 2012
mainly write written in python which is
the favorite language for everyone that
do programming develop developers and so
on
but the configuration file in in the
playbook actually use yamaha
yamaha itself is
very fairly easy to understand language
plus and simple also can start with a
simple architecture any one of the notes
that have ansible in store
can be the control or management node to
be used to manage the fleet of systems
and that's why i choose ansible
a few more other reasons i have chosen
ansible at the end is
is agentless you do not need to install
additional package on
the target notes you just require ssh
and python
as i mentioned earlier a yamaha is a
very descriptive language is so
descriptive and human readable and easy
to understand
that
i if i give you the code you mostly can
decipher what is supposed to be run
and you also support item potency
it ensure the same config is maintained
over
over the target notes no matter how many
time we rerun
and we run
and here i'm going to show you a simple
uh ansible architecture design that i
have done which is used in this we which
is going to use in the next demo
i have installed
ansible into my management notes which
is mgt and within the management node
you still need one more
key piece which is the inventory
usually it's a text file that have the
list of hosts that the ansible is going
to manage
and obviously we will need access
connectivity to our target host
and now
in this demo i'm going to show you how
to do it i have shown you how to install
apache web server with a shell script
and now i'm going to use the same shelf
script and run it with ncbi
in order to run ncbr i will need to have
ncba installed on my
control node mgt
which i'm going to install it wire yum
next i will need an inventory file
the inventory file
defines hosts and group of hosts which
the ansible will operate upon
for this i'm going to use the web server
text file that we have created as an
ansible inventory i'm going to create a
simple ansible playbook
which i'm going to call the same thing
the script
module as well as the script i'm going
to run
and we're going to run this
using
ansible playbook command
now
looking at this output
it actually tells us
task
reporting
it first he gathered the information
facts from
our target notes
and then telling us what he's doing is
setting up apache and there's something
changed on those boxes
and this is the summary report of what
happened
since we are running as a script it will
not be in item polden it will still tell
us something have changed
which
is part of the dilemma when we use the
module script
the true power of ncbo is when you are
using or utilizing ansible module
correctly
take for example
i have created
a myapp setup.yaml which is an ancient
playbook using the correct and simple
module instead of using script
in this new ncbi playbook
instead of using script module i'm using
other ansible modules such as package to
install relevant package for santos
and i'm using service to start the
service
http service for centos as well
as for installing apache in ubuntu i
need to install apache tool as well as
enable apache to service
to customize my index base page based on
the hostname i used
the template module
which utilize or convert
and simple facts or variable and lastly
i use the copy module to copy the files
from the management notes into the
target notes
for this i have created a new inventory
file
host and within this inventory file i
have grouped all three of my web servers
under a host group name web servers
we are going to run
our new
playbook
and against the inventory part
as you can see from this
and simple run
the output is much
nicer it tell you what you have done and
if it's already done for example php
have been installed you will just show
you okay
and if we have something changed for
example my template is different for the
index.html it will tell you something
have changed
and if i run this playbook again
you will notice that there's nothing
changed
anymore because everyone everything have
been sell according to our ansible
playbook
we have used ncbi playbook to sell up
and configure our web server
using
myself setup.yaml
now i'm going to show you how do we make
our
mys my setup.yaml
modular
i'm going to split my task into three
different rows
now let's go into rows and you will see
that i have three rows a rows that take
care of the part for ubuntu the rows
that take care of santos and a row that
take care of apache configuration
we will look at
our centos row
the main playbook here is under task
main
and in here you will see that i have
configured to have a
additional package to be installed for
centos and then i want to remove
unwanted package here
in apache
i will have
a
place where i need to install apache web
server for centos and apache web server
for ubuntu
as well as extra module that we require
for our web server
plus the customization of our
landing page as for this
um i will create i have created a new
playbook called site.yaml which will
call
each of the row
when it run
so for example i have centos that will
only include the center package and
ubuntu
this will be run for
all
hosts
but if you are part of
web server you will run
and set up apache
now let us
try out this playbook
against our existing host
and you can see that there's no error
and there's exchange
if we scroll up and see the output
you'll notice that
you will install
santos
stuff for only centos systems and you
install ubuntu
row when it's
detected is ubuntu but if it's a apache
row because this three servers is in the
web server's host group you will
install apache and configure
in the demo i show you how we can
quickly use the shell script in ansible
and then how do we quickly do a nc play
play and simple playbook before we
go into
ansible row
so what is ancieboro that i demonstrate
in the example earlier it's just a way
for us to develop reusable playbook by
grouping the component and encapsulating
the tasks the variable files
together
it actually helps
to
modularize and organize our ansible
playbook
but the good news is we do not need to
write all the ansible rules
by ourselves you can always go on to nc
by galaxy and search for other expert
written roles
you the easiest way that i find is go to
the ansible galaxy website and search
for the role or contain that you want or
if you want you still can run it via
command line
using ncbi galaxy search and the keyword
that you're looking for
as my favorite author in ansible galaxy
jeff gerling aka girling guy have
marvelous
role standard written in ancient galaxy
check him out
one last thing good about ansible galaxy
is it actually help you to have a
standardized
role structured when you
want to write your own ansible rows
i'm going to show you when you run
on the command line and simple galaxy
this is what you will find
which i find that is not really
uh interesting to look at
but if you search it on the website
this is what you're gonna find and you
will have a lot of more information this
is what apache and how many people have
downloaded and this is my favorite
author building
so far
i have only been using ancient on the
terminal
using command line
but with a great tool such as ansible if
you only do things via command line by
typing it
i believe
eventually we will have a lot of problem
because as a human we tend to type and
cause typo error you might be targeting
a wrong system because
you just target a wrong environment
instead of dev you go to pro
but
don't you worry there's actually an open
source project named awx which actually
provides a pretty decent i would say
nice web user interface for newbies that
does not
does not really fancy command line and
rest api for people that doing
development develop development and
devops
and you actually have to the task engine
is built on top of our assisting uh
ansible core infrastructure you don't
need anything extra and android x itself
will give you a centralized location
to manage the ansible playbook and
inventories
and now let me quickly show you the demo
how i use ansible
awx to deploy my apache setup
okay how we can import the ansible row
that we have created earlier into the
awx
this is my aws demo box
after you log in we should we need to
actually create a project
under project tab you're going to add a
new project i'm going to name my project
lca2
and sybil
i'm going to choose where the project is
going to get the source from i'm going
to get it from the github that i have
created
and i'm going to just save it
after you save
you can create inventory
inventory like i said earlier is where
you keep all your hosts
information your remote target
and for this i'm going to add a new
inventory
uh you can put anything here but i'm
going to say ncbi 2022
let's call it inventory
and i'm going to save it after you save
there's one important options here call
source and this is important because
this is
a feature that let us source from the
project that we have created earlier i'm
going to source from the project
and the project we have is lca 2022 and
silver
and i'm going to choose the inventory
file that i have kept in my git report
which is ancient host that we have been
using
and i'm going to name it
i'm going to name this source host
and i'm going to save
and after i save i do a sync
by doing a sync
awx will actually import the host
from
this project
into awx you will take a one
let's come to here there you go and we
will have our web server host group as
well as the three web servers
in the host group the next thing we need
to do is to create an
credential for remote login i'm going to
use
regram which is the default user that i
have in my vm and i'm going to set up as
a machine credential
with the username migrant and with its
password
in a production environment i do not
encourage you to use password please use
accessory private key for
login
the next thing in here you should
actually ensure that your previous
exclamation method is set to sudo
and we are going to save this
the last part for getting awx working is
we need to set up job template
in here we're going to add a new job
template and we are going to source it
again from our project that we have done
earlier
and i'm going to choose a playbook and
the playbook is site.yaml
and i'm going to have my inventory point
to
this guy
and to ensure i'm going to use very good
and i'm going to say this
is
my
lca
2022 site setup
after that is done make sure that you
click on privilege escalation
and if this is to meant to run
repetitively
enable fax storage as well that will
speed up in the future
run
save after you save you have options to
launch
the job
i will launch it now
the job complete without any failure
and we have a few changes in our centers
and ubuntu setup
if you scroll up further in the output
windows you will notice that the output
display here is quite similar to the
output that we saw when we run ansible
via the command line
one of the benefit that you can get from
the output is when you click on the
output
and you can drill down further of what
have been done
or run by the ncbi you will display in a
json format click on the
rocket to relaunch the job
and that will start the jobs again
when you run the job again for the
second time you will notice there is no
more changes because of the item button
behavior of n sebum
if you want to get more information of
how to use adwx you can always get the
help page
here
and that concludes my discovery journey
to find the right tools to help me to do
my configuration management i have
started doing shell scripting
ended up i
shelve it out and i use ancient mainly
to do my configuration management
and before i end the talk i actually
want to say thank you for
lca2 for providing this opportunity for
me to share this journey and all the
attendees and volunteers and tech
support who work hard for the success
i actually want to have a special shout
out to emily dunham because of her
speech you should speak that give me the
courage and encouragement to submit my
talk to lca
and special thanks to michael diehan for
his wonderful work in ansible and jeff
girling for sharing his ancient
knowledge which i bought a book from
if you have any feedback comments and
questions you can drop me an email send
me a tweet and i have my public github
that have contained some of my work in
ansible my try and error in puppet and
some of my containerizations are try out
i do have an infrequent update block in
wordpress which i update maybe once a
year but
that's it
thank you very much and i hope my
journey will kickstart your
configuration story as well
simon i can't hear you
luckily i was muted when i saw them
so someone asked wouldn't it be easy to
have the http the apache name as a host
bar and then use that to install the
package and service with the package and
service module
is i'm just giving a quick demonstration
of everything in the text very far the
correct way i would be doing is put it
as a variable and use that facts to set
it for different uh distribution or
operating system if you have different
package and for example you are going to
install ngix inside apache it's possible
okay
i think that's all the questions thank
you very much for your talk
um that's that is the end of our
conference today thank you for everyone
for coming along thank you to all the
speakers and thank you to the lca tech
people in our room and
elsewhere
and i hope you enjoy the rest of the
conference right