We provide analyst services. I’m an analyst. We are an analyst firm. You may have read or seen such statements on various web sites or information on profiles in Linkedin, but not everyone knows what an analyst is, what role they play in the industry, how vendors see them (for the most part) and how as an analyst and/or analyst firm can provide information to enable consumers to make an informed decision.
Who am I?
Who am I? That is a line from Les Miserables but as an analyst it can be an important foundation when dealing with vendors.
At my firm, we provide advisory (for vendors), consulting (for consumers) and analyst services. As an analyst (one of my hats that I wear), it is very common for vendors to reach out to schedule an analyst briefing. Equally, it is not uncommon for us to reach out to a vendor requesting an analyst briefing when we are seeking additional information, including product details.
Some vendors have an analyst relations department/team, with someone overseeing analyst relations. They handle all analyst requests from talking with the right person or people (depending on what information you are seeking), to scheduling briefings, which depending on the type of briefing could be focused on sales, financials and frequently, product including updates, rollout, functionality, and so on.
Independence
The vendor and analyst relationship one is quite tricky. On one hand there are vendors who feel that your role (as an analyst) is to present only positive information about their product and the company itself. If you find or identify problems or issues with the product or on how the company handles support or other items for example, there are vendors who feel that this is not relevant and therefore should not be provided to the consumer market.
I would like to make it clear that this is not a universal attitude among vendors in the e-learning space, but there are enough out there, that as a consumer you should be aware of, in terms of it existing.
I can’t speak for other analyst firms, but at The Craig Weiss Group, we are always 100% independent and as you know, I will always tell it as it is. If the product is good, then it should be identified and noted as such. If the product has problems or issues, then it should be known as well.
The same applies to the company itself. For example, let’s say you become aware during a briefing that the vendor is cutting their sales and support staff, due to financial constraints. As an analyst you will need decide whether or not this is relevant for people to be aware of, especially if you know the vendor is promoting that they are financially sound and everything is going great.
Or let’s say the vendor is showing on their web site an amazing system with lots of cool capabilities, yet when they show it to you, it is only 50% done and the rest still looks like it did from five years ago. The vendor is hoping as an analyst you present the cool features and user interface and keep the other, quiet. If you tell it as it is, you could face repercussions, if you gloss it over, the less likely the vendor is offended.
Repercussions
I have been very fortunate that the “we won’t show you anything” attitude is limited among all the vendors I come in contact with on a yearly basis. But it doesn’t mean it doesn’t happen. I can count on one hand, the vendors who feel that I am not presenting them in either a positive light or I did not present information accurately, when in fact I did.
This is part of where the expectation that you will write something positive about the vendor rears its ugly head. As mentioned early, there are vendors who, if they do not get what they want, will “blackball” the analyst.
Thus the analyst needs to make a decision on what they want or choose to do. As a firm, we never play that game, i.e. write only positive if we know there are also minuses. And as an individual analyst, equally, if there is something wrong with the system or the vendor, it is noted on the blog, my newsletter, reports, etc.
My personal feeling is that there are no perfect systems out there. Just as there isn’t a perfect person. We can all improve. Make enhancements, updates, tweaks, business direction and so forth to be better.
I’d rather have a vendor say, “yes, we are aware of that, ” or “we will investigate that”. My hope is in turn they will investigate that and get if fixed or have it on their roadmap to fix it at a later date (depending on the severity of the issue).
Quid Pro Quo
It may be a term you have heard or at least heard whispers of it. Another way to phrase it, is pay for play. Pay for play means that in return for a fee or amount of money, the analyst and/or analyst firm will repeatedly push out positive only information about the vendor’s product. Everything published by the analyst firm/analyst smells of fresh roses and happy land.
The vendor in turn, presents this information as proof positive that the product rocks and there is nothing to see here in terms of challenges, etc.
While I cannot speak for other analyst firms, I can state unequivocally, we never do quid pro quo or pay for play.
Even systems I love, I will always note the minuses if I find them. And again, there is no perfect system.
Embargo and NDAs
There are vendors who request an NDA prior to an analyst briefing. I never agree to them. An analyst has to remain fair and balanced, and if they hear information that they see as useful and beneficial to consumers, whether it is via a mini-brief or post, etc. then they should be able to provide that.
After all, many vendors seek input from the analyst, during certain briefings – related to the product. Some seek insight into their strategy overall, during briefing calls. Again, it adds to a slippery slope.
I will provide some feedback if asked upon but limit it. Not every analyst will – it just depends on the firm and the analyst themselves.
An embargo in the analyst world means that the information the vendor is providing has been agreed upon, prior to, that it will remain confidential until a certain date and time.
For example, the vendor may be launching a certain type of functionality that will be a difference maker (in their eyes). They tell you about the function and show it to you, but ask that you embargo it until X date. After they publish out on that date, you can, now present it to anyone.
I have no issue with an embargo, if it is agreed upon ahead of time. If a vendor says to me, “this is off-the record”, then I will not publish it, because it is confidential. Otherwise anything they present in a briefing is “on the record”. And I should note this is quite common in the analyst world.
On the record simply means that it can be published or presented to anyone. With an embargo it is delayed.
How to work with an analyst (attention vendors)
In the e-learning industry, the most common analyst briefings are with learning system vendors. Can it be with non learning system? Absolutely, but generally speaking, by far it is learning systems.
As well as I cannot speak for every analyst firm, nor will I try, here are some items that as an analyst, are useful to know and equally, not relevant.
I should note that if the call is covering financials and/or sales information then drilling down to the numbers and asking questions around that is a necessity. Vagueness is not what an analyst wants to hear.
For example, if you say, our year to year growth is 22%, then my expectation is you will give me the client numbers previous and current, so that it can be ascertained really how great or poor that growth is.
On the product side, a briefing should include
- A high-level PowerPoint deck. Focusing on the key items that either were agreed upon prior to, or that the vendor wishes to present that is relevant to the entire briefing at hand.
If the briefing is on the system itself, then a demo is a necessity, and because a demo can take time, having a slide deck that goes on for 20 minutes is not something an analyst wants to spend a lot of time on, especially if there is information that isn’t necessary for that call.
- A demo – either high-level or presents the information the analyst wants to know more about. For example, I will tell vendors I would like to see ABC as part of the demo. On the other side, a vendor may say, we will start by showing you are newest features, which is what I prefer, then we can go backwards.
Just the other day I had a briefing with a demo, and the vendor knew I was interested in metrics, so they focused their call on those new metrics they had in the system. Then, with time remaining, we were able to cover two other areas of interest. That is a strong use of time.
Pass..
Most analysts do not care who your clients are in a product briefing. Vendors love to show you because well, it shows who is using their system. While I can see that as useful information for a potential buyer, as an analyst, I don’t care.
Now, if we are in sales briefing, that is a different story, and that is why vendors know I will always ask them if they are the “exclusive vendor” for that client (which means they are the only learning system being used by that client).
Another I have no desire to care is where you have offices located. I can always find out that information on your web site or in a separate call. I’m happy for you that you have an office in Topeka, Kansas and a new one in Bournemouth, England, but from a product briefing, let alone a sales one, not needed.
What is your NPS?
I can tell you that vendors love to tell me their NPS (Net Promoter Score). NPS is a customer loyalty metric. I should note it should not be the only one you use, but that is for another time.
NPS scale is 0 to 100, with the ideal being as close as you can get to 100. Some vendors provide NPS with not only the system, but also their support too.
One of the items vendors never like to provide is the percentage of detractors (these are folks who will NEVER recommend the product, etc.). In NPS you have promoters (they score either a 9 or 10 on a scale which usually goes to 11), passives (score 7 or 8) and detractors (6 and below).
An NPS is an actual number on a range from -100 to 100. It is not a percentile.
Some Concerns
An NPS only takes into account, current- active customers, and assumes that as a promoter they will actually recommend the system/product to others, which may or may not be the case. How many times do you go to a hotel, they ask you if you would recommend them to others, you say yes, then never do?
Secondly, with recommendations it depends on who is presenting that information, for some people they seek experts i.e. in our case analysts, others it is Steve who runs the butcher shop at your grocery store.
Now Steve may know everything there is to know about a lamb shank, but do you want Steve to be your key recommender for a learning system? Most likely, you will thank Steve and then, ask where is the mustard aisle?
Bottom Line
As you can tell, being an analyst, in e-learning, isn’t the easiest job in the world. There is always something to consider. Something to ascertain. Something to question.
But when you do, and you do it independently, fair and without any quid pro quo, then it can be a very useful way to present findings in a manner that matters.
When an analyst says this is a top tier product or system or company, it should be your expectation that what they are providing is accurate, fair, independent and without any misleading information.
The only way to truly know is to ascertain the trustworthiness of the analyst and the analyst firm, thru previous insight and current knowledge on their focused market/industry.
If you are unsure, well, look for an analyst you can trust
Or ask Steve, the next time you are wanting to buy
a Roast.
E-Learning 24/7
