![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
canyonwalkerThis week, since returning to work on Wednesday morning after a relaxing long weekend in Phoenix, I've been developing a customer workshop. The deadline was short... It was decided early last week ago that I'd have to deliver it today. And not only that but I was on the road for 3 days last week with a business trip and then had 2 days of PTO planned this week. Thus the work in earnest on the creating the workshop began Wednesday morning. To run it for an audience of 6-8 devops engineers today at 9am. 😳
TBH I could have started before Wednesday. Although at the same time, not really. Mon-Tue were out because I was on vacation. Last week Thursday afternoon I'd just gotten back from a business trip and was tired out. And Friday I was both still tired out and had at least half a day of regular work to do. So I started this week Wednesday.
At first I was kicking myself for not starting until Wednesday. Despite the reasons above I kept scolding myself, "This would be so much easier if I'd done even 1 hour per day of it last Thu-Fri." That made me feel down. But at least I got going Wednesday. And I made steady progress at it, ultimately working until after 6pm because I was in the zone on it. That made me feel good.
Thursday was another "in the zone" day— at least up through lunch. After lunch I paused working on the workshop because I had a few important meetings to attend. When I got back to the workshop development work around 3pm I found that my infrastructure had gotten borked. 😰
My systems were wedged. About 95% of what I'd built was unusable. It wasn't anything I'd done; I hadn't touched it for 3 hours. And it seemed like it wasn't somebody else's fault, either; nobody else was using my infrastructure. Something had just failed on its own. And while I know enough about the infrastructure underpinnings to figure out, in broad strokes, what was wrong, I'm not expert enough in the details to have fixed it. Fortunately I still have one colleague— I used to have three, until the layoffs of a few weeks ago— who is familiar with the infra. Unfortunately it was already after 6pm for him and he was offline.
I messaged my boss to let him know. He was understanding of the predicament. He encouraged me to do as much as I could the rest of the day, and he'd help get my east-coast colleague involved at 9am EDT (6am my time) today. I finished the day Thursday pleased with how much I'd built in 2 days but crestfallen that a system snafu at the 11th hour might force us to push the seminar off to next week.
Well, I woke up at 5:40am today. It's not that frustration made me unable to sleep; just the heat. But the frustration was definitely there so I decided I might as well put on a shirt and log in to see if my colleagues were active. They'd just logged in a few minutes earlier. Logan, the guy who setup the infra originally, had just messaged that he was starting to take a look. Within 10 minutes he'd diagnosed the problem. It was basically what I'd figured out, but at a more granular level. He knew the specifics I didn't, so he knew which part had failed and how to reset it. He reset it, and within another 10 minutes everything was back up and running. The path was clear for me to deliver the workshop seminar today at 9am!
The seminar went well. We had a good number of attendees at the start, though half were called away less than 15 minutes later when part of their infrastructure went down. Oh, the irony! I even joked with them about my infra outage. But their infrastructure runs the e-commerce site for a small retail brand you might have heard of, so "down" for them means it's all-hands-on-deck for their infra team.
Their infra team was about half the attendees. The other half were apps people whose app was behaving just fine, so they could stay. I ran the workshop with them. It went really well. We ran until just after 12pm. The customer appreciated it, my sales counterpart said I did a fantastic job, and my boss really liked it, too. Good times!
TBH I could have started before Wednesday. Although at the same time, not really. Mon-Tue were out because I was on vacation. Last week Thursday afternoon I'd just gotten back from a business trip and was tired out. And Friday I was both still tired out and had at least half a day of regular work to do. So I started this week Wednesday.
At first I was kicking myself for not starting until Wednesday. Despite the reasons above I kept scolding myself, "This would be so much easier if I'd done even 1 hour per day of it last Thu-Fri." That made me feel down. But at least I got going Wednesday. And I made steady progress at it, ultimately working until after 6pm because I was in the zone on it. That made me feel good.
Thursday was another "in the zone" day— at least up through lunch. After lunch I paused working on the workshop because I had a few important meetings to attend. When I got back to the workshop development work around 3pm I found that my infrastructure had gotten borked. 😰
My systems were wedged. About 95% of what I'd built was unusable. It wasn't anything I'd done; I hadn't touched it for 3 hours. And it seemed like it wasn't somebody else's fault, either; nobody else was using my infrastructure. Something had just failed on its own. And while I know enough about the infrastructure underpinnings to figure out, in broad strokes, what was wrong, I'm not expert enough in the details to have fixed it. Fortunately I still have one colleague— I used to have three, until the layoffs of a few weeks ago— who is familiar with the infra. Unfortunately it was already after 6pm for him and he was offline.
I messaged my boss to let him know. He was understanding of the predicament. He encouraged me to do as much as I could the rest of the day, and he'd help get my east-coast colleague involved at 9am EDT (6am my time) today. I finished the day Thursday pleased with how much I'd built in 2 days but crestfallen that a system snafu at the 11th hour might force us to push the seminar off to next week.
Well, I woke up at 5:40am today. It's not that frustration made me unable to sleep; just the heat. But the frustration was definitely there so I decided I might as well put on a shirt and log in to see if my colleagues were active. They'd just logged in a few minutes earlier. Logan, the guy who setup the infra originally, had just messaged that he was starting to take a look. Within 10 minutes he'd diagnosed the problem. It was basically what I'd figured out, but at a more granular level. He knew the specifics I didn't, so he knew which part had failed and how to reset it. He reset it, and within another 10 minutes everything was back up and running. The path was clear for me to deliver the workshop seminar today at 9am!
The seminar went well. We had a good number of attendees at the start, though half were called away less than 15 minutes later when part of their infrastructure went down. Oh, the irony! I even joked with them about my infra outage. But their infrastructure runs the e-commerce site for a small retail brand you might have heard of, so "down" for them means it's all-hands-on-deck for their infra team.
Their infra team was about half the attendees. The other half were apps people whose app was behaving just fine, so they could stay. I ran the workshop with them. It went really well. We ran until just after 12pm. The customer appreciated it, my sales counterpart said I did a fantastic job, and my boss really liked it, too. Good times!
