• This year, Blink and Tesla Superchargers were added to the list of charging networks tested in 2020: Electrify America, EVgo, ChargePoint, EV Connect and Greenlots. 

• Electrify America earned the top score in the Digital Platform category, as well as the top combined score for both categories.

• Tesla Superchargers received the top score in the Charging Location Experience category—earning the most points in four of the five scoring subcategories.

Everybody’s talking about EVs these days, and more often than not, charging infrastructure is one of the first subjects that come up. Industry experts have different ideas about how much public charging will be needed, what forms it should take, and how it should be funded, but everyone seems to agree on the critical importance of reliable, convenient and affordable DC fast charging to enable widespread EV adoption.

For some years, Tesla’s proprietary Supercharger network was the only game on the highway, and many in the industry have called it the “gold standard” of fast charging. Today, drivers of non-Tesla EVs have access to a growing number of fast charging sites in most regions of the US. However, as Charged and other media have reported, the public charging experience often leaves much to be desired, and there’s a need for independent testing to help the EV industry focus its efforts to improve various aspects of public fast charging, including reliability, convenience, coverage and price.

Global engineering firm umlaut, part of Accenture, has been performing independent benchmarks from the user’s point of view in various industries for over 20 years. Last year, umlaut adapted its testing expertise and methodology to the US EV charging market and collaborated with Charged to publish the results of its 2020 USA EV Charging Infrastructure Benchmark.

Now, a year later, umlaut has conducted a second US study, this time in the Northeast. And this time, the study has added two more networks: Blink and the EV trendsetter Tesla.

For its 2021 EV Charging Infrastructure Benchmark report, umlaut conducted a comparative study of the largest fast charging networks and rated each of them using several criteria. umlaut looked at interoperability, pricing, transparency, functionality and availability, among other elements. How fast, and how simple, is the charging process? How convenient are the charging locations? Is the charging power that’s delivered the same as what’s advertised (which is dependent on the state of the charge of the EV’s battery)?

What was tested: methodology

As they did for the 2020 benchmark, umlaut’s team set out in 2021 to test and compare the user experience at some of the best DC fast charging sites on each of the tested charging networks. Only top-rated sites were chosen, based on data from PlugShare—a free app (acquired by EVgo in July 2021) that allows users to find and review over 140,000 charging stations in North America. So, to be clear, this is a comparison of the charging industry’s best efforts: a search for the best user experience currently available.

umlaut’s testers covered 2,100 miles to find and test top sites from each network—through 7 states over 6 days during September 2021. The testers used a Ford Mach-E and a Tesla Model 3 as test vehicles, and conducted nearly 150 tests at 28 stations (4 each from 7 different networks) in Michigan, Ohio, Maryland, New Jersey, New York, Connecticut and Pennsylvania.

This testing methodology is tailored very specifically to the user’s perspective of the charging networks. Also, this year the team used a device called a PLC sniffer to trace the communications between the vehicles and the CCS charging stations to add some technical performance metrics to the scoring system (not applicable to Tesla Superchargers). 

In order to rank the different networks, umlaut split its analysis into two areas: the charging network’s Digital Platform (website and app); and the Charging Location Experience. The company developed a comprehensive set of key performance indicators (KPIs) in order to compare the most important aspects of charging among the different networks.

Category 1: Digital Platform

Website
Especially for new EV drivers, a company’s website might be the first point of contact. The following questions are important for users:

• Is there a tutorial and knowledge base to demonstrate to new EV drivers how charging works?
• What types of network memberships are available?
• How transparent is the pricing?
• What route planning features are available?

App operation
The focus in this category is the user interface of a network’s smartphone app. Criteria considered include:

• ease of use
• intuitive operation
• ease of account creation
• charging instructions
• ease of locating available charging stations
  

App functionality
The capabilities of smartphone apps were considered as a separate category. umlaut took a detailed look at the functionalities of the various networks’ apps, including:

• real-time data
• reservation options
• route guidance
• filtering options (charging power and plug types)
• tech support options
• information about charging power and time
• language options
• visibility of stations belonging to other networks
  

Price transparency and payment
Whereas gas stations display their pricing on prominent signage, at EV charging stations it is often not clear what the cost of a charging session will be. The umlaut team evaluated:

• price structure
• price transparency (ability to see pricing before, during and after charging)
• roaming (ability to use multiple networks with one membership)
• available payment methods (credit, debit, PayPal, Apple Pay, Google Pay)

Category 2: Charging Location Experience

Charging station environment and features
The physical surroundings and amenities of a charging station are important. umlaut’s testers considered the presence and quality of features including:

• signage
• illumination
• ease of parking
• roofing/weather protection
• charging connector options (CCS, CHAdeMO, Tesla)
• charging power available
• amenities (restrooms, shops, restaurants)
• free WiFi
  

Tech support
From instructions to authentication and payment to technical problems, there are several issues that can disturb a successful charging process. umlaut contacted the networks’ support hotlines for help with various simulated scenarios, and scored the results based on the visibility and availability of the hotline number, multi-language options, error access and any charging tips provided.

Access and payment
umlaut awarded extra points for networks that support Plug & Charge, a new system that authenticates a network member and initiates charging as soon as a vehicle is plugged in, with no need for an app or RFID card. The testers also considered the range of payment options available at the actual charging site, as well as (naturally) the price of charging.

Technical performance
Does the charging process start on the first attempt? How long does it take to initiate charging? How does the actual charging power delivered measure up to what is promised?

Here’s what was not tested

• This is not a test of the feasibility of long road trips while using any of these charging networks. Not all of the tested networks are available nationwide, and this is not a comprehensive overview of the US charging market. Instead, we’re focused on the user experience at the charging site and while using the apps and websites.

• This is not an ownership-experience benchmark of the EVs used in the tests—a Ford Mach-E and Tesla Model 3. No data or metrics from the cars were measured, and the EVs did not affect the scores earned by the charging networks (which made it tricky to include Tesla’s closed-loop EV-to-Supercharger ecosystem in the study—more on that below). There are countless vehicle reviews and owner-satisfaction surveys for EVs online, so if that’s what you’re looking for, you’ll have no trouble finding it. 

Apples-to-apples: How do we include Tesla?

Last year, a lot of people asked, “Why didn’t you include Tesla Superchargers in the benchmarks?” The answer was simple: umlaut set out to develop a scoring methodology to compare the user experience of public charging networks, and many of those scoring categories didn’t apply to Tesla’s proprietary closed network.

This year, umlaut tried to adapt those scoring categories to fairly account for Tesla’s proprietary nature in a way that would produce the most useful data to compare it to open networks. That’s easier said than done, however. 

The first obvious challenge is that Superchargers can only charge Teslas, and there are a few scoring subcategories that are affected by this, including the Connector Type metric. “This might sound like an automatic negative for Tesla, but it isn’t necessarily,” explained Christian Sussbauer, one of umlaut’s EV experts who led the study. “We do not simply count the number of EV models with different connectors—that methodology wouldn’t make sense. Instead, we looked at the market share, how many EVs with the different connectors are actually on the road, and what percentage each charging network can serve. That’s actually quite a bit for Superchargers because the number of Teslas on the road in the US is 75% to 80% of EVs.”

The bigger challenge with including Tesla in the 2021 Benchmark is that none of the open network’s scores are affected by the vehicles being used to test them. So if the Mustang Mach-E’s dashboard or navigation displays a bunch of useful info while charging at a ChargePoint station, that doesn’t affect ChargePoint’s score in any way. All the other open networks are only scored by the data available on their apps, websites and charging station displays. So, to include Tesla, all the data and info available on the vehicle was ignored as well, and only the Supercharger locations, along with the charging info available in Tesla’s app and website, were considered.

This will probably be the most controversial part of the methodology, as Tesla drivers will be quick to point out that one rarely uses Tesla’s app or website in the same way one would with the other networks, because most of that useful charging and navigation info is elegantly built into the vehicles. But, again, that’s not what was tested here. This is not a comment on the Tesla ownership experience, it is an attempt to compare the DC fast charging networks in the most useful way. To do so, Superchargers were considered a standalone charging network independent of Tesla vehicles.  

The challenge with scoring these metrics is revealed in the Digital Platform category results, where Tesla Superchargers lost a lot of points because the smartphone app and website are not as useful as those of some of the other networks, regardless of what Tesla vehicles display. However, as you may have read, Tesla is plotting a course for all EVs to have access to its Supercharger network, and plans to use its smartphone app to make that happen. So, at the very least, these results can serve as a useful guideline for Tesla on how to improve its app and website for all the future EVs that will use Superchargers without the benefits available in Tesla’s closed vehicle-charging ecosystem.

If you have questions/comments about the scoring methodology, please let us know:
best-in-test@ChargedEVs.com

Network ratings

Last year, Electrify America was the clear winner of umlaut’s Best-in-Test award for the USA Charging Infrastructure Benchmark. This year, it was a closer race. EA once again won in the Digital Platform category with a score of 275, but ChargePoint ran a close second with 270. Tesla Superchargers, which were not included in last year’s rankings, handily won the Charging Location Experience category with a score of 484. However, as Tesla’s app and website lack a lot of useful info, Superchargers earned a dismal score of 165 in the Digital Platform category, and Electrify America once again took the overall prize with 702 points. Tesla Superchargers came in second overall with 649, followed by ChargePoint (611), EVgo (578), Greenlots (548), Blink (505) and EV Connect (472).

“We had a lot of internal discussions about including Tesla Superchargers in this year’s Benchmark,” said umlaut’s Christian Sussbauer, “because we really want to create a testing system that will capture the right metrics and provide the most useful information to the EV industry. To be honest, if you asked me before we conducted the tests, I think I would have predicted that Tesla Superchargers would end up leading in total points because it’s well known that they do a great job in many aspects of fast charging. And actually, our test results show this clearly—Superchargers excel in many important areas. It’s actually a little boring to test Supercharger stations because you just plug them in and they reliably work.”

“We were happy to find a very high level of simplicity and reliability with Electrify America this year as well,” Sussbauer continued. “Using an EV with Plug & Charge enabled on Electrify America’s network was a very good user experience. All of the EV industry should strive to replicate that aspect of Superchargers and Electrify America/Plug & Charge. And in the end, Electrify America also has a very useful smartphone app and website that combined to put them over the top in terms of total points scored for the metrics we were measuring in 2021.”

Electrify America

“Electrify America delivers an increasingly consistent and positive experience at its charging locations.”

Electrify America overall score: 702

Pros:

• Plug & Charge available
• App is much improved, and is easy to navigate
• Many high-powered stations (up to 350 kW) along highways
• Illumination makes charging stations easy to find at night
• Solid, high-quality website
• New feature displays kW requested by EV and kW provided by EVSE

Cons:

• Relatively high prices
• No roaming possible with other networks
• Charging process takes a long time to start
• No RFID card available
• No QR code scanning for charging station identification
• Support hotline waiting time was more than 10 minutes in some cases
• Up-front payment required to create account
• No multi-stop travel planning on website

Tesla Superchargers

“By far the best charging locations and environment, and the connection between Tesla’s EVs and Superchargers is state-of-the-art.”

Tesla Superchargers overall score: 649

Pros:

• Automatic authentication and payment (same functionality as Plug & Charge)
• By far the fastest to start and stop charging process
• Huge number of stations along highways (up to 250 kW)
• High-quality website and app: glitch-free

Cons:

• No charging for non-Tesla EVs
• Charging costs not shown in the app
• App has limited charging station map functions
• App doesn’t show service hotline number
• Hotline not always available, or long waiting times (over 10 minutes)

ChargePoint

“Charging stations always worked perfectly.”

ChargePoint overall score: 611

Pros:

• Roaming with some other networks
• Great new display and positioning of stations
• Very silent EVSE
• Most other networks’ stations shown in the app
• Good payment options
• Displays in 6 languages

Cons:

• Plug & Charge not available
• Not many high-power stations
• No lights at some charging locations
• Sometimes difficult to reach EV inlet with cable
• Account required to see charging map and pricing
• Hotline number not displayed at all locations

EVgo

“EVgo is the only network that supports all current US DC fast charging standards.”

EVgo overall score: 578

Pros:

• Roaming with some other networks
• Support for all US charging standards (CCS, CHAdeMO, Tesla)
• Consistent price structure
• App provides easy access to nearest charging station on home screen

Cons:

• Plug & Charge not available
• Not many high-power stations (most are 50 kW)
• Charging stations in poor condition compared to other networks (dirty, faulty displays, etc)

Greenlots

“App experience is the best among all contenders—account creation is very fast.”

Greenlots overall score: 548

Pros:

• Best roaming options among all contenders
• Comprehensive real-time data during charging (kW, kWh, SOC, time)
• Charging station identification is quick from the app
• Hotline service is also provided in Spanish

Cons:

• Plug Plug & Charge not available
• Often, an unnecessary additional step is required to start charging (Start button on the station itself)
• No payment options besides RFID and app
• Many chargers have no proper display
• Price information missing
• Not many high-power stations
• Website experience needs improvement
• Process to start and finish charging takes a long time

Blink

“The website is very informative.”

Blink overall score: 505

Pros:

• Notification when a charging station becomes available
• Great experience with service hotline

Cons:

• Plug & Charge not available
• No roaming with other networks
• Not many high-power stations
• Too few payment options at charging locations
• Pricing very high—sometimes $10 minimum
• Prepayment required to create account
• No filters on website
• No filter in app for power levels (kW)
• One display got so hot that touchscreen didn’t work

EV Connect

“Good charging experience—easy and convenient.”

EV Connect overall score: 472

Pros:

• Hotline service is good
• Good charging station locations
• Prices are indicated during charging process

Cons:

• Plug & Charge not available
• Maintenance of charging stations needs to be improved
• Very little information provided on website
• Unsatisfying experience with the app
• Number of successful charging attempts was low—only 1 out of 4 stations worked consistently
• Charging start process takes too long
• Real-time data is not always correct

Conclusion and outlook: 2021 key findings

The overall US charging network is growing quickly, and the customer experience is improving. Newer public charging stations are delivering a better charging experience. umlaut found clear improvements in reliability and interoperability (roaming) among networks compared to last year’s tests.

Capacity seems to be sufficient for today’s volume of EVs, at least in the regions where umlaut conducted this year’s tests. Popular charging sites in some of the country’s EV hotspots may be crowded, but umlaut’s testers experienced no wait times during their 6-day testing process.

Plug & Charge, a new system (part of the ISO 15118 standard) that makes the charging process much more seamless and convenient, is the talk of the charging world these days. Unfortunately, it may be a while before it’s implemented on a widespread basis—umlaut’s testers were able to use Plug & Charge only at Electrify America stations. (Tesla’s Superchargers have always used a proprietary system that offers the same features.)

While the EV charging experience is steadily getting better, there is still considerable room for improvement, especially when it comes to networks’ digital platforms (websites and apps)—this category had the largest disparity among different networks.

Interoperability, or roaming—the ability to access different networks with a single membership, as one does with cell phones—is still limited. Electrify America, Blink and Tesla offer no roaming capability. Most networks don’t even show the locations of other networks’ stations in their apps. Drivers still need a third-party app (such as PlugShare) to locate all available charging stations, and the integration of charging into popular navigation apps such as Google Maps is poor.

At charging sites, drivers will often miss the convenience they were used to in the old gas station days (though not the smell). It’s sometimes unclear what the charging price is, or what charging power level is on offer. Waiting times to start and end charging with CCS are too long compared with Tesla. And many charging locations lack amenities such as food, restrooms and weather protection. When things go wrong, calling a service hotline can be a frustrating experience—wait times often exceed 10 minutes.

Charged and umlaut plan to continue this collaboration to bring you future reports on the state of the charging network user experience. The next benchmarks will expand to new North American markets in the United States and Canada.  

The testing experts: umlaut
Q&A with Hakan Ekmen, Managing Director umlaut

Comments?
You can reach the testing team with comments, clarifications, suggestions for future work, etc. at best-in-test@ChargedEVs.com.

This article appeared in Issue 58 – Nov/Dec 2021 – Subscribe now.