How Are We Improving the Quality of Deltek PIM to Support Your Business?

You have probably read or heard my boss Mike Scopa talk about quality improvements within Deltek Engineering and our “shift left” philosophy. This is a global Deltek Engineering effort, so the Deltek Project Information Management (Deltek PIM) engineering team is very much part of this process. 

In this post, I'll share how the engineering team is working to improve the product quality of Deltek PIM to better support AEC industry businesses.

Software Development Life Cycle

Before I get into the details, it is important to explain how software is created, which is known in the software industry as the Software Development Lifecycle (or SDLC for short). Here is a diagram that shows this process:

• The Requirements Analysis phase is where we decide what features we want to include in this release.
• Design and Development are where the developers actually write the software, based on overall architectural design principles.
• Testing is the formal quality assurance process to ensure that the new software features are functioning correctly and that the existing features are still working as planned.
• Maintenance takes place after the software has been released to our clients, and is where we fix any bugs that have slipped through to the final version. This also includes work to keep the software compliant with third parties and other changes in the marketplace.

All of our work is carried out by 1) product managers who represent the customer and define the release scope and 2) engineers (developers and quality engineers) who carry out the work to deliver software that meets the product manager’s goals.

Why Shift Left?

The best thing we can do to help customers is to produce better software all in an effort to decrease bugs affecting our customers and to release more software features more quickly.

We looked at various studies across the software industry and the key message was that the cost and effort of fixing bugs rises exponentially throughout the software development life cycle (SDLC). This approach is similar to the AEC industry where it is easier to fix a design issue on paper than it is to fix that same issue during construction.

To address this, we are now applying the “shift left” concept, putting as much effort as possible into finding bugs early in the SDLC. This is all good in principle, but is meaningless unless it is put into practice. That is why we have implemented a few practices that can help: 1) Code Analysis and Review, 2) Security Scanning and 3) Testing.

1) Code Analysis and Review

Code Analysis is an automatic process that is built into each developer’s toolset (e.g. Microsoft Visual Studio). As they write software, the analysis runs and highlights areas that infringe the rules. The goal is to avoid common coding mistakes and enforce a consistent style to all of the code. The latter is an important quality factor because many bugs will appear more obvious when the code has a consistent style.

Code Review is a manual process that is carried out in two stages. The first stage is known as a peer review because the developer simply shows the code to one of their peers, who examines the code and asks questions. This step often highlights obvious problems that may have been missed by the original developer.

A more formal code review is carried out with the developer submitting the software for testing. This step is a “gate” within an automated process – so the software cannot get into the test build until it has been approved by one or more senior developers. During this review, the senior developer checks the code rigorously for styling issues and checks the functionality for possible errors. Any comments that are made have to be addressed by the original developer who then re-submits the software and the changes are checked before the software can be submitted for testing.

The Deltek PIM engineering team has been using these processes for several years, but we have enhanced this during the past year to include the automatic analysis tools and to further define how the manual code reviews should be run.

2) Security Scanning

We use several levels of security scanning during our software development processes. It is also worth mentioning, that we discuss new software features with our internal security teams during the Requirements Analysis phase to head off any potential security problems before we write any software for those features.

All of our software goes through a security analysis tool, which analyses the computer code and searches for known security vulnerabilities. This produces a live list of issues which are then assessed and either marked as “false positive” or raised as a defect and then added to our development backlogs.

We also have specific security testing servers on which our security team runs manual tests to identify any vulnerabilities. Any vulnerabilities found at this stage are raised as defects and added to our development backlog.

Finally, Deltek PIM is subject to periodic external assessment by a third-party consultant. This does not happen for every release but is done in line with standard Deltek policies. Anything found at this stage is raised as a defect and added to our development backlog.

3) Testing

Traditionally, software is tested at the end of the development phase, using a single large test plan which tests all of the software manually – known as regression testing. Any bugs found during this phase are costly to fix and can endanger the release date for the software. As part of our ‘shift left’ initiative, there are a number of testing areas that we have been working on.

Automated Testing

In modern software development, regression tests are still used but the manual processes are replaced by automated tests, which are mini programs that run parts of the software and check its behavior. Typically, these automated tests are run more often during the development phase. The frequency varies across projects depending on the time required to run the automated tests.

This is an area that we have made a lot of progress in during the past year, including the addition of a dedicated test automation specialist to our team. We track progress on this month by month and aim to have all of our Maintenance Release testing done automatically by the end of this year, at which point we will be extending our automated testing to cover our full regression process.

Developer Testing

Having the automated testing in place is a positive step towards improved product quality, but any bugs raised are still relatively costly to fix. Therefore, we are also pushing as much testing into the actual development process as possible.

We have adopted two strategies for this. The first is called “Over-the-Shoulder Testing” where the developer has an informal meeting with one of our quality engineers to demonstrate the software and check if it meets the acceptance criteria. The developers often invite the product managers to these sessions as well, which allows them a chance to see how the software holds together before it goes into the formal testing phase. Any issues that are picked up at this stage are dealt with by the developer before the software is submitted for testing.

The second strategy is “Unit Testing” which requires the developer to identify areas of the software that would benefit from specific internal testing. For example, if there was a module that calculated tax on an amount, a test could be written to apply different tax rates on different amounts and to verify the results are correct. These tests then become part of the overall build process for the software. If a test fails, it is impossible for the developer to submit software for formal testing without first fixing the issue. This type of testing is designed to catch inadvertent changes that could break internal functionality (and therefore cause problems that you, the customer, would see). This is the latest area of shift left activity for Deltek PIM, but we are increasing our test coverage each month and will continue to do this for new functionality and select parts of the old.

Conclusion

Hopefully, this description of the Deltek PIM team’s shift left initiatives shows how committed we are to software quality. I personally have been leading the PIM engineering team for 10 years now, and we have evolved our practices throughout that time. But this last year has definitely been the most accelerated period of improvement. These new shift left practices will continue and we will continue to evolve and improve our practices in future years. I am confident that all our customers will see improvements over time, but we are always open to feedback about specific areas that our customers would like us to focus.