When I tried pair programming for infrastructure work, it turned out to be super effective!

This content originally appeared on DEV Community and was authored by kakisoft

In one project, I explained infrastructure task to assign other team member, while sharing my screen. The situation ends up similar to pair programming.

Surprisingly, I fount it is hugely effective. So, I thought "It is the most effective method for infrastructure tasks"
Then I will tell you how to do it.

What is pair programming?

Pair programming is a method that two engineers program using only one computer.
One acts as coder, and another acts as supporter. The role can be switched when it's needed.

The problem of pair programming (In my opinion)

Each engineer often has different develop environment. As times go by, we have more and more variety of choices to build our develop environment.

For example, hardware such as the number of the display, type of keyboard, and mouse.
And software such as editor, the configuration of shortcut key and shell.
There are uncountable numbers of differences.

It is impossible to prepare develop environment customized for you when do pair programming, because you have to share hardware and software with your partner.

I use VSCode, default shell without any customization, a few alias, a few snippets, some plugins.
If my partner needs Vim, or heavily customized shortcut keys for their performance, we lose productivity.

Fir of all, these days most of engineers work remotely, so it is not common to work at same place.

Considering these points, it is difficult to prepare not only to keep their performance but also incorporate this method.

How does pair programming works for infrastructure

Unlike normal programming, it is common for engineers to have completely different infrastructure environment.

Once they login to server, there is no difference of their useful command.
Also, it is rare to have a necessity to work with many displays.

So, you can make pair programming environment only by sharing engineer's display and using chat application.

It is easy to prepare online as well.

Benefits of pair programming for infrastructure development

I found these benefits.

Reduce obscuring of infrastructure works

Infrastructure works tend to be obscure, because it is common to see only one person for infrastructure work.
And the number of infrastructure engineers is often reduced due to budget.

Also, many issues are left out with excuse of "we will follow up".
Some cases, the project have many kludges to work around unnecessarily complicated infrastructure.
These information and knowledge are not officially documented and shared with team members.

These knowledge tends to be exposed when other engineer take over from resigning engineer.

But, you can find out inside of closed box in pair programming, because when you work for infrastructure with sharing display, the partner can ask "What is that for?".
Also, the team can make sure what should be documented after the task.

It is possible to take over without formal documents

Infrastructure engineers often have to do ineffective task because of unoptimized infrastructure.
And, documenting those takes away the time and effort.
(ex. Previous assigned engineer did inappropriate job, which is confusing and complicated. The deadline was close, so the team did last minutes job, thinking to redo it. But that last minutes job stayed for a long time.)

It is not rare to get stuck in a negative cycle like putting off documentation later because it requires a lot of effort, not being able to find other engineer to manage infrastructure, taking forever to educate other engineer because of how complicated to explain.

But, if you work infrastructure task using pair programming method, you can explain about complex issue when the partner ask you "Why do you use this operation?".

Many infrastructure task and documents are not reviewed unlike source code.
Pair programming can solve this problem.

It has the same effect as code review by pair programming. You can find more effective approach from your partner.

All infrastructure work can't be done by code even though IaC exists.
So, pair programming let you work productively.

Notice commend mistake

There are a lot of infrastructure task that you need to pay attention.

Some tasks are nerve wrecking because one small mistake on command often lead to critical problem.

But, when you write a command with shared screen, you can ask your partner to check.

If you made a mistake, your partner can warn you, and reduce your mistake.

Share useful commands

You will use many variety of commands at infrastructure works.

Some commands that you use on a daily basis make an impression to other engineers like "I didn't know that command." or "This is new way to use this command for me."

Also, you can share some information like "I should use this command when I want to check it"

Collaborate to solve the errors

If you face unknown error, you can divide the task to investigate.

For example, a engineer check the logs from the web server such as nginx, the other engineer check the logs from application such as Laravel.

You can get an advantage by using same server. Because it doesn't mean you both checking different local data.

Good for your mental health

Personally, it is the most effective point.

Since our infrastructure tasks have a significant impact on project, you get ill if you get unknown errors and you have no idea how to solve it.

For example, you might face a situation like,
"Shit... I just deployed as usual. But why am I getting this errors?? I have to solve this before the demo tomorrow morning."
"Are you serious? It is already 4:00 AM!?"

In this case, by researching with your partner, you can reduce your stress and get more effective.

I highly recommend this to engineer like you who have got stomachache by infrastructure task.

Our actual conversations

It is not easy to imagine these advantages by only reading. So, I wrote down our actual conversations.

I changed some phrases to make them easier to understand, but the general outline is almost same as actual one.

case 1．

Me: "Oh! I got an error in npm install. Why? Have you added some new libraries?"

My colleague: "Yes. I have added some of them. They might be the problem."

Me: "OK. I'll check the error message. I found it. Oh! The problem comes from Node.js v17."

My colleague: "Why do you use v17? The development environment uses v18."

Me: "Amazon Linux 2 is not supporting v18. So, I installed v17 unwillingly."

My colleague: "Really? Is it impossible to install v18?"

Me: "Yes. We need to prepare Amazon Linux 3 to use v18. At first I was using Amazon Linux 3. 3 is too deferent from 2. So, I changed 2."

My colleague: "Is is so different?"

Me: "Definitely. We can't use scripts and commands from Amazon Linux 2 in Amazon Linux 3. We had to relearn so many commands.
I can't understand why it is necessary to change drastically. I swear to god not to use Amazon Linux any more.
The deadline for demo environment was just around the corner. I didn't have enough time to relearn them. So I discarded 3. Then I tried to install v18, but I couldn't."

My colleague: "Then, we got this shit."

Me: "Oh, but v16 might work. Let's install nvm and change the version."

case 2．

Me: "Oh! Now, I got a different error in npm install! Why? What's wrong?"

My colleague: "I have no idea. I'll check it now."

Me: "Thank you. I'll check it too."

• investigating -

My colleague: "We get this error when the Computer doesn't have enough memory, maybe this is the problem."

Me: "Really? micro is not enough. I remember we were using most of memory in previous project."

My colleague: "We can't think of other reason. Let's change the setting and try it again."

Me: "OK! I'll try."

My colleague: "By the way, have you change the EC2 setting? I am scared because I have never done it."

Me: "Don't worry. I have done it before. It was not complicated and quick. You can only stop the instance and change the setting."

• working -

Me: "Oh. It's not working."

My colleague: "We got same error. Let's upgrade instance size more."

Me: "Sure. Let's try medium."

• working -

Me: "Yeah! It's working now!"

My colleague: "We finally fix it."
f
Me: "I avoid unstable version of Node.js. I think it is best to use latest version in case of Laravel, but Node.js is not like that.
I will only use Node.js LTS version. Also, I will stop being stingy for memory and use medium for EC2 "

case 3．

Me: "Shit! I got seeder error!? What's wrong??"

My colleague: "It works in local."

Me: "I execute it after refreshing tables, so it is not possible existing data causing this. And it works in our local environment. What's wrong?? I have no idea!"

My colleague: "I'll check that."

• researching -

Me: "I might find it! Look!"

My colleague: "Oh, you found it? ... what the fuck is this!?"

Me: "There are 2 seeders! One of the seeder is "Out*DoorUnitGroupSeeder", and other one is "Outd*oorUnitGroupSeeder".
The one has uppercase "D", other one has lower case "d"."

My colleague: "Why do these files exist in a same directory??"

Me: "I have no idea. Maybe, a developer pushed a file with wrong name. He realized his mistake and re-pushed it with correct name.
If this assumption is right, it is my bad, I should have checked better."

My colleague: "Which is correct?"

Me: "Upper case is right. It is enough with that."

My colleague: "What is going on.? There is only 1 file in local machine."

Me: "Maybe, we both use Windows. Widows can't distinguish upper case and lower case of file name.
Then, when we run git clone, Windows controls it.
So, we have only 1 file, and we can execute without errors."

My colleague: "I see.
....By the way, how to fix it? Windows only have 1 file, so we can't fix it."

Me: "I'll use Mac and push it. Give me a second.
We could edit the source code directory in EC2. But, it might cause annoying task at next deploy.
It is not difficult task, I'll do it."

case 4．

Me: "CORS error never disappear! What's wrong!?"

My colleague: "We have investigated everything we could come up with. Why?"

Me: "We have demo for the client tomorrow. Why we face such so many errors only today!"

My colleague: "What do we do with it?"

Me: OK! Let's disable CORS."

My colleague: "Are you sure?"

Me: "Yes. I am sure. No problem. I fix it immediately after demo. It is too late, we have to focus on making it work rather than investigating."

My colleague: "How do we disable CORS?"

Me: "We can only comment out this middleware. we gotta pull through."

My colleague: "Really? you edit the source code directly?"

Me: "We can't include this. I'll write down we must execute revert before deploy."

My colleague: "It is too late. We don't have much more options."

Me: "...OK! Maybe it will work."

My colleague: "Hey! I found an another error!"

case 5．

My colleague: "I can't log in! What's wrong!?"

Me: No idea. I found that user is registered.

My colleague: "Let me see... Oh! I found the error."

Me: It looks like some backend errors.

My colleague: "OK! Let's investigate together. I will check nginx logs. Please check Laravel logs."

Me: "Sure!"

• investigating -

Me: "I got it! It is a permission problem. It couldn't write laravel.log."

My colleague: "Great. Please fix it."

Me: "OK! ...Yeah! I could log in!"

case 6．

Me: "Phew. We finally finished the deploy. The frontend app works with PM2, and it is stable now.
Hey! Look at what time we started talk. It has been 4 hours."

My colleague: "I'm exhausted."

Me: "Thank you for implementing PM2. I did't tell you, I needed to run this using "npm run dev"."

My colleague: "Are you kidding?"

Me: "No. While the demo app running, I restarted it manually when some errors happened. I had my console on side screen."

My colleague: "What were you doing? lol"

Me: "I didn't have other options. Complain to your frontend leader if you have any. I didn't have time to care about it.
He is always on unimportant tasks, and run away from critical issues. Every single time he found tiny error and insist that "it it serious problem".
I'm struggling to get his progress."

My colleague: "I have heard about it. I don't know the details because I joined the project recently."

Me: "He dump all frontend technical issues on me. Actually, it didn't even build until last week.
I told him that we have demo soon, but he let it cooked."

(It was a total chaos. I wrote why this happens in this article. To sum up, we had an engineer who might fake his CV. )
【システム開発】タチの悪いエージェントの地雷を踏んだ時、被害を最小限に抑える方法を考えてみた

Summery

It seems we are having extremely pointless conversations, but if you have worked as an infrastructure engineer, you have faced similar situation without letting other members know.
Right? I know you have.

Like that, I believe it is very effective to use pair programming in infrastructure jobs.
When I take over this kind of job, I will ask them "I want you to explain it with sharing screen, because of these reasons. "

This content originally appeared on DEV Community and was authored by kakisoft