Let op! Internet Explorer wordt niet meer ondersteund. Hierdoor kan de website mogelijk niet goed functioneren, gebruik een alternatieve browser om optimaal gebruik te maken van deze website. Klik hier om een alternatieve browser te downloaden.

TIOBE QA Award Q4 2022

6 January 2023

Philips ULTRhy wins the Q4 2022 Software Quality Assurance Award

TIOBE rewards the best TiCS projects with the quarterly TIOBE Software Quality Assurance Award. There are more than 7,700 software projects of various multinationals worldwide that qualify divided into 3 categories:

  • large-sized projects (more than 500K lines of code)
  • mid-sized projects (more than 100K lines of code)
  • small-sized projects (less than 100K lines of code)

TIOBE Software Quality Award Q4 2022
The winner of the most ambitious award, the large-sized projects, for the fourth quarter of 2022 is …

ULTRhy (Philips – Bothell USA)

The complete top 5 of large-sized projects

Projects are (out of 409 projects):

Q4 2022 Q3 2022 Project Company TQI Score TQI Level
1. (1) ULTRhy Philips Bothell USA 92.53% A label dark: Top quality software
2. (2) Eleva System Control Philips Pune IND 91.40% A label dark: Top quality software
3. (3) Platform Markem Imaje Keene USA 89.04% B label: High quality software
4. (4) OceanusRfDConsole Philips Bangalore IND 86.81% B label: High quality software
5. (5) IGC Philips Best NL 86.47% B label: High quality software


The winning ULTRhy team of Quality Award Q4 2022

The winners of the mid-sized projects

Projects are (out of 1,604 projects):

Q4 2022 Project Company TQI Score TQI Level
1. Michi TomTom Amsterdam NL 98.16% A label dark: Top quality software
2. NxGen Philips Bengaluru IND 96.85% A label dark: Top quality software
3. User Workflow Philips Bengaluru IND 96.69% A label dark: Top quality software

The winner of the small-sized projects

Project is (out of 5,756 projects):

Q4 2022 Project Company TQI Score TQI Level
1. Playground Mastervolt Amsterdam NL 99.99% A label dark: Top quality software

Winner mid-sized projects Q4 2022 - TomTom Michi

The Winning ULTRhy Team

ULTRhy controls embedded imaging systems handling the acoustics, signal and image processing, user experience, patient and image management, media, connectivity, service, and administration of the devices it serves.

For this occasion, we have interviewed Bob Desmarais, who is the lead architect of the ULTRhy platform. He is responsible from concept through delivery of first product.

What kind of software quality process do you have in place? Can you elaborate a bit on this?

Advancing the efficiency and quality of the embedded software for Philips imaging products ranks as important as the acoustics, signal and image processing. We summarize the goal as half the code with twice the efficiency of existing platforms, and we exceeded both.

Before writing the first line of code, we set standards for software development and adopted a strategy that each standard must be measured by tooling to demonstrably show we met the standard. After experimenting with several tools we selected Resharper from Jet Brains for static analysis, Squish Coco from Frog Logic for unit test coverage, and MS Test for the unit test execution engine.

We invested in platform tooling for automated integration testing, event recording with automated script generation for creating clinically accurate test scenarios, C# based scripting to extend the recorded script for stress testing, randomization, and fault injection. We built dynamic analysis into the platform for performance and resource monitoring. Failing to meet an internal quality check leads to error log entries and error log entries are treated as automation test failures. We expose and address problems early, before they are allowed to pile up.

What practices do you recommend to others to achieve high quality software?

  • Instill a high-quality bar in the technical leads who demonstrate quality with every review and every delivery; the team rallies behind them.
  • No rule without tooling; do not rely on peer review.
  • Build quality checks into the software and report an error when they fail; treat errors as failure in your automated tests, both unit and integration tests; these cannot be ignored.

What role played TiCS in achieving better control on software quality?

Two roles. First as a view into the current state of the evolving system. You can optimize only what you can see. Watching code coverage increase and duplicate code decrease motivates right behavior. Second as a source for tools. Given this is a new platform, we did not expect to see much code duplication and were surprised by the duplication numbers reported. We have integrated PMD code duplication tooling into our build and it is checked with every delivery by our CI machinery.

Do you have any suggestions how to improve TiCS to make sure you are doing even better?

There are two problems we ran into. First the internal connections for the Coverity server proved unstable and remained so for months; there were extended periods where we could not rely on the TiCS dashboard as the results bounced around daily. The second is the speed with which TIOBE vendors such as Coverity and Parasoft support language features. We ran into analysis issues as we adopted the latest language features supported by our compiler vendors: Intel and Microsoft. As an aggregator, TIOBE is able to help apply right pressure to vendors.

Previous Winners

  • Q1 2022 – Q4 2022: ULTRhy Bothell (USA)
  • Q4 2020 – Q4 2021: Philips Eleva System Control Pune (IND)
  • Q3 2019 – Q3 2020: TomTom NavKit 2 Amsterdam (NL)
  • Q2 2018 – Q2 2019: Philips Eleva System Control Hamburg (D)
  • Q4 2017 – Q1 2018: TomTom NavKit Amsterdam (NL)
  • Q1 2015 – Q3 2017: Philips HealthTech Eleva System Control Hamburg (D)
  • Q2 2014 – Q4 2014: Philips Healthcare Eleva UI Hamburg (D)
  • Q3 2013 – Q1 2014: ASML OTAS Eindhoven (NL)
  • Q1 2010 – Q2 2013: Philips Healthcare Eleva UI Hamburg (D)
  • Q1 2009 – Q4 2009: Océ Technologies VarioPrint Venlo (NL)
  • Q1 2008 – Q4 2008: NXP Semiconductors STB225 Southampton (UK)

Interested how TIOBE can help you?

Contact us and find out how TIOBE can help you gain software quality.

Contact us