Introduction to Amazon CodeGuru

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Introduction to Amazon CodeGuru:Automate Code Reviews and Application Performance Recommendations

Srinivasan H Sengamedu Daniela Tzvetkova

Senior ML Manager Senior Product Manager

Amazon Web Services Amazon Web Services

A day in the life of Lynn

• Lynn is tech lead working on Java projects in an ecommerce company

• part of a distributed development team

• responsible for the backend services (search, order, and shipping) of her company’s high volume

site

• Her responsibilities span the entire application development and operations cycle

• D: We found a data corruption issue in production.

• L: Let’s find the root cause.

D: I think it is due to a data race.

Could we have caught it during code reviews? I wish we had someone who really understands concurrency.

• O: The site latency is increasing. I just got paged!

• L: Let’s find the root cause.

O: The CPUs are overloaded. Can we increase the fleet size?

• L: We increased the fleet size last month. The traffic is pretty much the same. What’s going on?

• O: ???

• L: OK, let’s increase the fleet size.

How do we find out what’s actually going on? I wish we’ve a performance expert in our team!

What’s on Lynn’s mind?

How can we improve code

quality?

Are we giving lowest latency to our customers?

Are our infrastructure

costs just bloating?

Lynn’s ecosystem

Write +

Review

Build +

TestDeploy Measure Improve

What’s missing in Lynn’s ecosystem?

• Detection of code defects early in the cycle

• Keeping up with coding best practices

• Identifying performance bottlenecks and linking them to code

• Tools for visualizing application performance

• Availability of expertise

• Faster time to resolution and remediation

• Developers need a truly integrated tool.

• The tool should provide actionable recommendations across phases in the life cycle.

What is Amazon CodeGuru

Amazon CodeGuru Reviewer

Amazon CodeGuru Profiler

Demo

Introducing Amazon CodeGuru

• Machine learning service for automated code review and application performance profiling

• Trained on decades of knowledge and experience at Amazon

• Evolves with user feedback

• Searches for optimizations continuously, even in production

• Provides actionable recommendations to fix identified issues

• Automatically inspects code for hard to find defects

• Helps you find the most promising methods for optimization in your running application

It is like having a distinguished engineer on call 24x7

Amazon CodeGuru: Using ML to Code Review and Optimize High-Performing Applications

Easily identify performance

and cost improvements in

production environment

CodeGuru Profiler

Detect and optimize

the expensive lines

of code pre-prod

Built-in code reviews

with actionable

recommendations

CodeGuru Reviewer


• Provides automated code review comments

• Supports Java applications

• Integrated with GitHub and AWS CodeCommit source code repositories

• Leverages Pull Request-based code review workflow

Pull Request-based Code Review Process

1. Lynn creates a branch.

2. Lynn makes code changes.

3. She creates a Pull Request.

4. Code reviewers provide

comments. Lynn provides

responses.

5. The code changes are merged after approval.

Pull

Request

Approval

Merge

Code

Review

Branch

Make

changes

locally

Code Review Key Challenges

• Expertise• Availability, Compliance and Correctness aspects often do not get addressed because of

lack of expertise.

• Senior Talent• Code reviews often demand a senior engineer to be involved. Teams may not have the

right individuals or they may be focused on other high value tasks.

• Multiple functional areas• The number of topics which require expertise, e.g., AWS API use and concurrency, is

increasing

• Human code reviews often focus on business logic and less on functional correctness.

• Number and size of source code repos increasing• Reviews often require inspecting a large amount of source code for context


Flags critical defects and reliability issues in source code.

Amazon CodeGuru Reviewer augments human code review process and does not replace it

Pull

Request

Approval

Merge

Code

Review

Branch

Make

changes

locally


Code Areas addressed by CodeGuru Reviewer

AWS Best Practices: Correct use of AWS APIs

Incorrect use results in performance (e.g., polling) or correctness and completeness (e.g., pagination) issues.

Concurrency: Correct implementation of concurrency constructs.

Incorrect use results in correctness (e.g., missing synchronization) or performance issues (e.g., excessive synchronization) and hence impact availability.

Resource Leaks: Correct resource handling

Incorrect handling (e.g., not releasing database connection) results in slowdown and impacts availability.

Sensitive Information Leak: Leakage of Personally Identifiable Information

Leakage of sensitive information (e.g., logging of credit card number) leads to compliance issues.

Code defects discovered by mining data: Hard to find defects

Correcting issues (e.g., not creating a client for each lambda invocation) improves code quality.

CodeGuru Reviewer Workflow

Code

Repository

CodeGuru

Reviewer3. Recommendation

4. Developer

Feedback

1. Repository

Association

Repo Admin2. Pull

Request

Developer

Amazon CodeGuru Reviewer – How it Works

Customer performs

Pull Request

Input:

Source Code

try (GZip gzip =

GZIPInputStream.create(

url.openStream())) {

use(gzip);

} catch (Exception e) {

handle();

}

Extract semantic

features/patterns

Feature Extraction

gzip =

GZIPInputStream

.create(stream)

use(gzip)

ENTRY

EXIT

stream =

url.openStream()

gzip.close()

handle()

throw

Exception

ML algorithms + Program

analysis identify code defects

Machine Learning

Code

corpus

Customers see

recommendations as

Pull Request comments

Output:

Recommendations

Amazon CodeGuru Reviewer Demo

CodeGuru Reviewer – Concurrency

public String get(final String ip) {

if(!IP_PATTERN.matcher(ip).matches()) {

return ip;

}

if (repo.containsKey(ip)) {

return repo.get(ip);

}

…

}

Code

Recommendation“repo” is a ConcurrentHashMap and your usage of containsKey() and get() may not be thread-safe. In between containsKey()

and get(), another thread can remove the key and the get() will return null. Consider calling get() and using its result.

Good catch of a potential race.

Developer Feedback

public String get(final String ip) {

if(!IP_PATTERN.matcher(ip).matches()) {

return ip;

}

str = repo.get(ip);

if (str) {

return str;

}

…

}

Fix

CodeGuru Reviewer – Concurrency

synchronized (orderObject) {

obj = orderObject.get(name);

if (obj == null) {

obj = new orderObjectMarkdown(name, category);

orderObject.put(name, obj);

}

}

Code

Recommendation

Correct.

Developer Feedback

synchronized (orderObject) {


if (obj == null) {

obj = new orderObjectMarkdown(name, category);

orderObject.putIfAbsent(name, obj);


}

}

Fix

CodeGuru Reviewer – Looping vs Waiting

This code appears to be waiting for a resource before it runs. You could use the waiters feature to help improve

efficiency. Consider using TableExists, TableNotExists. For more information,

see https://aws.amazon.com/blogs/developer/waiters-in-the-aws-sdk-for-java/

Recommendation

do {

DescribeTableResult describe = ddbClient.describeTable(new DescribeTableRequest().withTableName(tableName));

String status = describe.getTable().getTableStatus();

if (TableStatus.ACTIVE.toString().equals(status)) {

return describe.getTable();

}

Thread.sleep(10 * 1000);

elapsedMs = System.currentTimeMillis() - startTimeMs;

} while (elapsedMs / 1000.0 < waitTimeSeconds);

Code

We should use waiters instead - will help remove a lot of this code.Developer Feedback

Fix:

https://aws.amazon.com/blogs/developer/waiters-in-the-aws-sdk-for-java/

Feedback from Amazon Developers

“Amazon CodeGuru Reviewer allowed us to catch a long-standing race condition lurking in a critical piece of code”

“We were using an Amazon DynamoDB table in a way that we don’t typically use and we missed this in the code review. Amazon CodeGuruReviewer identified that the operation was returning paginated results instead of all results. Great catch and great recommendation.”

“Amazon CodeGuru Reviewer recommended I use a try-with-resources to close the resource, and I implemented it immediately. +1.”

Key challenges with poor application performance

• Poor end-user

experience

• Rise of performance problems:

Troubleshooting of distributed applications is

challenging

• Not enough performance engineers: Scarcity

of performance engineering expertise

• Higher cost of

compute

infrastructure

• Impact on mission

critical systems

Business impact Causes & contributing factors

• Losing customers

• Performance optimization is challenging:

not a domain expertise for most developers

CodeGuru Profiler finds your most expensive lines of codein your live application

• Trained to find methods with high-potential for performance optimization• High latency & low throughput

• High CPU utilization

• Recommends how to fix your code

• Intelligent profiler trained by many years of performance engineering experience at Amazon

• Interactive visualizations

Built for production systems

• Low overhead (~1%)

• Continuously runs on production

• Continuously analyzes performance

• Main use cases• Troubleshoot a bottleneck• Proactively improve latency and cost• Performance test• Learn how your application works

• Currently supports applications written in Java

Onboarding

Create Profiling Group

1

Update the IAM role used by the Profiler agent

2

Set Java application dependencies

3

Start Profiler agent in your application

4

Amazon CodeGuru Profiler Demo

How it works

CodeGuru Profiler – How it Works

Customer’s application

Profiler thread


Profiler thread


Profiler thread


Profiler thread

Actionable recommendations

Amazon Developer Feedback on Profiler

Chris Butterfield, SDE

CodeGuru Profiler’s recommended fix removed the thread contention which was using 55.97% of CPU time. After the fix a single host could now serve ~7.5x more traffic than before. We reduced our number of instances by ~75% while still handling the same traffic

Rajesh Konatham, SDE

After following Profiler’s recommendation to remove these clones, we saw huge reductions in CPU utilization – a 40% reduction on the synchronous fleet and 67% reduction on the asynchronous fleet

Next steps

• Onboard CodeGuru Profiler

• https://aws.amazon.com/codeguru/

• Download the sample app

• https://github.com/aws-samples/aws-codeguru-profiler-sample-application

• Start profiling!

https://aws.amazon.com/codeguru/

https://github.com/aws-samples/aws-codeguru-profiler-sample-application

Summary

• Amazon CodeGuru makes it easy to improve code quality and application performance

• Amazon CodeGuru continuously learns and improves.

• Amazon CodeGuru Reviewer

• Identifies code defects. Provides actionable recommendations on code fixes

• Easily integrates with pull-request based code reviews

• Amazon CodeGuru Profiler

• Actionable recommendations on performance issues

• Rich visualizations to help troubleshoot issues

Get started at aws.amazon.com/codeguru

Thank you!


Srinivasan H Sengamedu Daniela Tzvetkova

Introduction to Amazon CodeGuru

Documents

Transcript of Introduction to Amazon CodeGuru