Last week, Veeam and Nutanix have published a joint technical paper, regarding the best practices to use Veeam Backup & Replication v7 in a Nutanix infrastructure, when running VMware vSphere. The paper is 20 pages long, it has been written by myself and Derek Seaman, and you can download it here.
I’m not going to explain in further details what’s in that paper, It’s really detailed and gives you all you need to configure these two technologies and have them work together at their best. This post is about what’s “not” written in that paper, its story, and what I learned from this great experience.
It took me and Derek, plus several other people (back on this later…), more than a month from start to finish. The final result is “only” 20 pages, but the emails we exchanged, the phone calls, the webex sessions connected to lab environments, all those would fill a small book if put on paper. This document is the final version of so many drafts, some with only tiny adjustments, other with huge changes we did from a draft to the next.
The first lesson I learned is that working in the hyper-connected world is really nice, but at times can be a nightmare; or at least, incredibly time consuming. I was involved as Veeam technical leader in this project, and I live in Italy. Together with me, I had people from Veeam based in the US East Coast (6 hours behind me), our developers in Saint Petersburg (3 hours ahead of me), Derek in San Jose, CA (9 hours behind me), and Derek also had some colleagues from New Zealand (10 hours ahead of me) reviewing the drafts. If you try to place everyone on a map, I’m sure we would have been able to follow the sun for an entire day. Sounds cool? Not when you have to exchange informations, review logs, test different parameters, and your main communication tool is email, because it’s the only way to manage such an asynchronous communication. I’m sure we would have nailed the same document in one full week if we could have been in the same room. But that’s the world as we know it today, so it’s ok. And is nice to work from home with people all over the world.
The second lesson is about test procedures “in a real world”. We started with some theories about what the best possible configuration would have been, and we tested it throughly in the lab. We adjusted several options and parameters, but after a couple of weeks we realized that simply that configuration was fascinating, but not secure enough for our common customers, and could have lead to problems. So, we decided to drop it, and we started again with what originally was a “Option B”, but it has become the final best practice at the end. I learned a lot more about how things work inside Veeam Backup & Replication, and I also got new informations about Nutanix. And I learned that when it comes to make two technologies work together, it’s a slow and complicated activity to find the right balance, to adjust configurations, and sometimes to discharge previous best practices because they do not work in a totally different scenario.
The third and final lesson I learned is about the people. You may know me as one of the “public faces” of Veeam, and even if Derek has just joined Nutanix he’s from time a known blogger, and a VCDX. However, this paper is the work of many people that do not appear into it. Speaking about Veeam, there is a guy from support who helped with log analysis, configuration errors, additional tests in our Support Labs, and also found a damn typo in a configuration key (shame on us…) that was creating weird results. There’s a solution architect who gave us suggestions coming from his large experience on the field, and he also used a PoC with a Veeam/Nutanix customer to help us with “real life” tests. And there is a guy from the developers’ team who reached us to further analyze logs, and to explain us some behaviours and “internals”, and how to better configure some parameters. Those guys are usually “in the shadow”, they are not listed in documents nor they attend public events, nonetheless their effort in this activity was incredible, and I’m sure we would have not be able to complete the paper without them. Their names are Cody Ault, Tom Sightler, and Rustam Kovhaev, and they deserve all the credit like us.