Running a proof of concept minimises the risk of an unhappy client, so why aren’t they being used more?

Undergoing a Proof of Concept (PoC) for a client is nothing new. The security of a ‘test run’ of new equipment makes sense for all sides. And clients, especially new or particularly nervous (or indecisive) ones have often insisted one takes place. And they’d be right to do so. With big money on the table, it gives the client the peace of mind they are getting what they paid for, at the same time it gives the consultant/integrator the reassurance what they specified for the client is going to be a good long-term investment and will probably secure future business. So it is probably surprising to find out that PoCs are not taking place with the regularity you’d probably expect, given what we just outlined.

Inavate Magazine spoke to a selection of consultants and integrators to get their take on PoCs, to find out how they approach them, what the issues in doing them can be, and why they don’t happen every time.

Inavate began by asking if they did a proof of concept every time they started a new project, and the consensus was that if the project involved kit the consultant/integrator hadn’t used before, then the answer was definitely yes. If it was a brand new client it was more likely that one was done, but if it was existing kit and a current client, then it was unlikely. As this response from Anthony Lear, design engineer, pre-sales department, Kinly, highlights; “We do PoCs on a kit basis, but we also do them on a client basis as well. We have our own tech lab for testing and proof of concept equipment. We make sure it does what it says on the tin essentially.”

Lear adds there are two levels of testing for Kinly, one for new equipment and one to see how multiple devices from different manufacturers equipment work together. AV consultancy Hewshott changed its approach to PoCs four years ago explains managing director Dan Lee. “We realised there was a problem with the process that all consultants use, which is that we do requirements gathering, we design systems, we recommend them to the client, the client goes off and buys 100 of them. And then it’s not until the first one goes in, or the integrator does the first one of the off-site tests that the client actually gets to see it working.”

Hewshott’s new plan was to insert a step where it did a proof of concept test before the client actually bought any products. “Before it goes out to procurement we borrow all of the kit from the various manufacturers that we’ve specified. And we mock up the system off-site. What we found really interesting is it’s not so much about testing that all the components work together, because we design systems that work generally, it’s more things like how does the client’s very specific locked down laptop build respond to manufacturer X’s little gizmo? For those are the unknowns you don’t find out about until you walk into the first room, and then you realise you got a massive problem, because you’ve ordered 100 of these things, and you now need to change 100 of them.”

Project AV’s approach to PoCs might seem a little bold at first sight, but there is a lot of acumen behind its policy, as COO Steve Hudson explains. “We don’t send boxes out, because they get kicked in a corner, or they don’t get set up correctly. So we say to people this is the cost for a PoC, we’ll come and install it, we commission it, we pre-stage it, we’ll manage it, and we’ll put you on our support portal. We treat it like an install.

We give it to the customer and leave it with them between 30 to 60 days. I take a PO (purchase order) from the customers for the whole room. At the end of the 30-day PoC if they don’t want it, we rip up the PO and we’ll come and take the proof of concept away. Do you know how many I’ve had back? None, because it’s done properly.”

The idea behind issuing the PO, rather then just loaning a potential client kit with no financial commitment attached, is that it weeds out clients who are serious about investing in an AV project says Hudson, rather than just someone who wants a free demo of the latest products to have hit the market recently. What happens if you don’t do a PoC, are the results barely noticeable, or is disaster likely to strike?

A milder version of the latter says Jack Cornish, technical director, Tateside, but in going without one, you must be prepared for challenges along the way. “We’re always looking for the same end result, ie a happy client, but getting to that result might be a bumpier ride if we’re not doing a full PoC.” There are two things that will happen if you don’t do one says Lee, added costs being the obvious one.

“It’s definitely more expensive to change things after the fact, ie to make a wiring change in a rack during a rack build, which takes 20 minutes in total to build, takes two hours on site. You multiply that by 100 rooms and that’s a huge chunk of time and cost. The other thing is sometimes no matter how good you are, it’s actually quite hard to explain exactly how these systems are going to work. What off-site PoC testing allows you to do is bring the client in and actually walk them through what you’re proposing.” Lee’s point here is an excellent one, and one that all AV suppliers should be asking themselves. We can tell buyers how great product XYZ is, but for non-technical end users that information may as well be in a foreign language sometimes.

With a PoC you can see in real time what it does. “You can take 20 hours of conversation and turn it into 20 minutes of demonstration, to get to the same level of understanding on the client side. It’s very powerful,” explains Lee.  As designers, we all design systems that should meet the client’s requirements says Lear. “But as great as a PoC is, it all depends on where you do that testing. We can design a PoC of a system off-site, but when you’re designing a system, you need to look at the room and the environment it’s going into and how it all fits in. We can never  guarantee it’s going to work where it’s going to go unless the PoC is in the room it’s going to end up in, and we’ve connected with everything we need to connect to.”

This leads us to the next conundrum regarding PoCs, if it’s difficult to replicate the environment the kit is eventually going in to, does that mean a PoC loses value, or becomes redundant altogether? “PoCs can’t always be the definitive be all or end all. But I think it gets us 90% of the way there,” says Cornish.

For bigger projects Project AV and Hewshott have both created a replica of a client’s ideal classroom or boardroom in their own offices, and the client has come into their offices to get a look and feel. For smaller projects, like Microsoft Teams Rooms (MTR), Project AV likes the PoC to take place at the client end. “With MTRs we tend to rather than it was with them. They understand the process of everything. And it also gets every stakeholder to take a look at it.”

Time is a critical factor when discussing PoCs. We all know integrators are time poor, so is a PoC something that can disappear from the schedule if a project gets squeezed? Size matters here says Cornish. “It definitely adds time to a project, if it’s a huge project it becomes much more palatable than on certain smaller projects.” It also differs from client to client adds Lear, “How much value does a client place on it, is it important? Yes. Does it get dropped at times? Yes. Some clients will just want stuff in now. Especially with some of the lead times (seen recently), people don’t want to wait.”

The supply chain crisis is definitely having a knock-on effect on PoCs. Some we spoke to for this article reported manufacturers not having any kit to send for testing, and other manufacturers had such little kit they wouldn’t sent out test units unless a PO was signed first. The shortage of kit is being felt by all  says Hudson, “I’ve been sent kit by a manufacturer who said I can borrow it for three days. How can a PoC be done properly (in that time)? They don’t know what room it’s going in, what the end user wants for that room.”

Manufacturers are also sending kit direct to end users, and its not being sent up properly, which is then affecting a final decision on projects adds Hudson. When one is run, how much importance do consultants and integrators place on a proof of concepts for the overall success of a project? Or does the fact they are sometimes culled from the project altogether answer that question for us? “No one wants to hear at the end of a project ‘Oh, but I thought it could do this’,” says Cornish. And that is the heart of the matter, a PoC helps keep clients happy, or at least fully aware of what their money is buying.

For Lee, it’s about two issues; confidence and risk, “What a PoC does do is allow us to go to tender, instruct a contractor to install all the bits and pieces, much more confident in the knowledge that we’re not going to require any last minute changes, or we’re not going to find anything at the off-site testing that we weren’t expecting, or that doesn’t work as it should. We do lots of thingsalong the way for our clients that are about risk mitigation. And  this is just another one.”

Finally, we asked our experts how widespread they thought proof of concepts actually were in the AV industry, the answer was about one in every three projects. Which means they are in the minority. “I don’t think POCs are used as much as they probably should be. People are just trying to design equipment and get projects installed these days a lot more than they used to,” says Lear.

It’s a regretful situation concludes Hudson, “If you can do a PoC it’s key to the client understanding that isn’t just the right kit, it’s understanding how the kit set up, what they need to do in the background, and what’s required from them at the end of it i.e. how do you hand this over to your staff? How do you make sure everybody can use it? Too many are saying ‘let’s throw some kit in’, they’re not thinking of the bigger picture.”

Credit: Paul Milligan, Editor – Inavate Magazine

If you’d like to discuss a Proof of Concept for your next audio visual installation, contact us today!

images:
Maha Heang 245789/shutterstock.com
naytoong/shutterstock.com