Have you ever broken a system at work?

Oh man! Ad-blocking software has been detected! :'(

This website is run by the community, for the community... and it needs advertisements in order to keep running. Blocking our ads means your killing our stats!
Please disable your ad-block, or become a premium member to hide all advertisements and this notice.

Today I was flashing the BIOS of a machine, which I do alot of. Whilst flashing this system it started to give unusual beep codes , I started to panic, wondering if I had installed the wrong BIOS or whether the process had gotten interrupted. I started to look for an open window just incase I needed a quick get away.. luckily it all went well but the thought crossed my mind “what if I had broken it, what would happen?” Wondering if any if you had been in a situation where you caused more harm than good?

 

Oh man! Ad-blocking software has been detected! :'(

This website is run by the community, for the community... and it needs advertisements in order to keep running. Blocking our ads means your killing our stats!
Please disable your ad-block, or become a premium member to hide all advertisements and this notice.

Blew up a signal generator once... If you smell burning, then you're doing more harm than good.

 

One day around lunchtime, I was doing maintenance on the servers for a small bank. The servers handled ALL the transactions for the bank - the main office and two branch offices.

So the APC software showed some error messages and it suggested that I run a test on the rack-mounted UPS. Okay, sure, I'll run a test. So I clicked the Start Test button.

You know that sound when a server spins down? The servers in the room were all plugged into that UPS... and they all made that sound simultaneously. I froze, because I knew that every bank teller PC in all three of their banks had just locked up.

Worse, the server room was right across from the CEO's office - and he was looking right at me.

He heard the sound and saw me spring into action, so he came over and asked me what was wrong. I told him what happened, and he smiled and shrugged. "No big deal. It's lunchtime."

Imagine my shock and surprise. You'd expect a CEO-type to be furious. Nope. And at that moment, I was grateful that he was so understanding.

I got the servers back up and running and we immediately ordered another rack-mount UPS. I popped open the case of the old one. Turns out the batteries had swelled up huge. Could have done a lot worse damage if they had exploded.

 

Once I screwed up WSUS policy and caused about 40 production servers at 40 retail sites to update and reboot on sunday evening. Thankfully impact was small, sites were not too busy.

One place I worked one of the admins deleted top level OU with hundreds of users, groups, PC's underneath. Think Microsoft had to help with recovering that...

Done some other mistakes that made me feel warm quickly but don't feel the need to admit all It's generally good learning experience, once you screw up few times you will be wary to do changes that could have any effect on the business. You will also be able to sense when your colleagues don't investigate impact enough... simply because you've already been there!

 

Changed my settings to view history items in our main program. When asked the question to apply to all I click yes.

Spent all day going round to users manually setting it back to how it was, or close as possible if they couldn't quite remember! Still to this day don't know why I clicked yes knowing the ramifications.....

 

You know, there used to be an issue with older APC units where plugging in a standard serial cable from a windows system that was polling the serial port for a mouse would shutdown the UPS pretty much instantly
they sold a special cable that removed the pins that APC didn't need, that caused the issue, but you know, charged a pretty penny for it so everyone kept using standard DB9s
Never happened to me but I know a few colleagues who crossed that rail.. yikes!
------
I had just started at a hedge fund as a new Cisco/NT bod and had to ghost my own machine when the equipment arrived
Asked the manager about the switch config and if multi-cast was properly configured 'sure, its what we always use'

So I proceed to ghost the system while working on my temporary system
anyway, a few minutes later people start coming in asking if there is a network problem, and my machine seems to have locked up (you know the whole mouse thing not moving and such)

Wouldn't have put two and two together so quick but as soon as the progress bar on the ghost machine reached 100 I picked up my mouse cursor 'unlocking' and moving all over the screen from the pending commands I must have issued

I had just dumped a 28GB image to every port on the network simultaneously