Bio 130 tissue culture --intro lecture dev strategies part i tissue cultue 2013-2014
Do we need a bigger dev data culture
-
Upload
simon-dittlmann -
Category
Engineering
-
view
105 -
download
0
Transcript of Do we need a bigger dev data culture
![Page 1: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/1.jpg)
Do we need a bigger DevData culture?
![Page 2: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/2.jpg)
Abstract
In front-end software development it's still rare that data is collected on the client side besides some analytics data where developers usually don't have access to.
Imagine what you can do when you have front-end log data, you see how many ajax calls hitting your servers and you finally know whether the single page application is used like you expected or not. I will briefly talk about projects I was part of where we used these kind of data to improve our product and surprisingly reduced AWS costs by changing front-end code.
![Page 3: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/3.jpg)
Who am I?
Simon DittlmannSaving the world from toil since 1998
Developer/Ops, technology nerd@SinnerSchrader #IoT, #docker, #js, #frontend, #infrastructure, #kubernetes
Contact SimonDittlmann github.com/Pindar dittlmann.com
![Page 4: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/4.jpg)
DevData
Collaboration between Dev, Ops and Data (Science) to collect useful data to improve your
software continuously.
![Page 5: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/5.jpg)
DevData: Why and How?
Everything fails all the time. Quality assurance testing is not enough. Let’s formalize our developer assumptions, monitor them with data and improve.
Tools: so many. Google Analytics, Elastic-Stack, Datadog, Loggly etc.
![Page 6: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/6.jpg)
Use cases
A. Logs: Client Side Exception trackingB. Metrics: API performance, Call-Count, Page PerformanceC. Application tracking: Find the important to prioritize
![Page 7: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/7.jpg)
Logs: Client Side Exception tracking
![Page 8: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/8.jpg)
Logs: Client Side Exception tracking
Assumption: if there is no JS error in my developer console there aren’t any at all.
So often wrong. For example private browsing is totally different implemented in various browser.
How to log client exceptions the easiest way without integrating yet another tool and without hitting my own infrastructure? Use e.g., Google Analytics Exception tracking.
![Page 9: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/9.jpg)
Logs: Client Side Exception tracking with Google Analytics
window.onerror = function (msg, url, lineNo, columnNo, error) {
ga('send', 'exception', {
'exDescription': [
'Message: ' + msg,
'URL: ' + url,
'Line: ' + lineNo,
'Column: ' + columnNo,
'Error object: ' + JSON.stringify(error)
].join(' - ')
});
};
![Page 10: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/10.jpg)
Logs: Client Side Exception tracking with Google Analytics
![Page 11: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/11.jpg)
Metrics: API performance, Call-Count, Page Performance
![Page 12: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/12.jpg)
Metrics: API performance, Call-Count, Page Performance
● Monitor how long an API call takes both on the server side and on the client side
● Monitor the amount of calls throughout a day● Program with edge cases in mind
![Page 13: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/13.jpg)
Monitor Server Logs
● Check for latency, response time etc.● Monitor which endpoints are requested the
most.● Do load tests from time to time, e.g. with
$\> ab -n 100 -c 10 -C \ $cookie_string $domain
● Take advantage of state of the art tooling:○ Elastic Stack (aka ELK): Kibana,
ElasticSearch, logstash○ GCloud Stackdriver and BigQuery○ AWS Cloudwatch logs○ Software as a Service Solutions like loggly,
papertrail…
![Page 14: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/14.jpg)
User Timings: Tracking an API call
var t0 = performance.now();
ga('send', 'timing', 'api1calls', 'start', Math.round(t0));
fetch('https://www.example.com/testApi1')
.then(function(response) { return response.json(); })
.then(function(data) {
var t1 = performance.now();
var timeItTook = t1 - t0;
console.log(timeItTook, JSON.stringify(data));
// Sends the timing to Google Analytics.
ga('send', 'timing', 'api1calls', 'load', Math.round(timeItTook));
return;
});
![Page 15: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/15.jpg)
User Timings: Tracking an API call
![Page 16: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/16.jpg)
Polling data might be dangerous during deployments
![Page 17: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/17.jpg)
Do not accumulate ajax calls
var pause = false;
setInterval(() => { // the interval is usually shorter than the http timeout…
if (pause) return;
pause = true;
fetch('https://example.com/testApi').then((response) => { return response.json(); })
.then((data) => {
console.log('do something');
pause = false;
}).catch((error) => {
console.log('There has been a problem with your fetch operation: ' + error.message);
pause = false;
});
}, 5000);
![Page 18: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/18.jpg)
Disable polling when page is not visible
// This may save you money in an autoscaling environment
// functionality uses vendor prefixes, see documentation for details.
document.addEventListener(visibilityChange, handleVisibilityChange, false);
function handleVisibilityChange() {
if (document[hidden]) {
clearInterval(timerId);
} else {
timerId = startPolling();
}
}
// https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API
![Page 19: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/19.jpg)
Application tracking: Find the important to prioritize
![Page 20: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/20.jpg)
Optimize the important
● Use analytics data to know what to optimize● Collect the right data: to support that use html data-* attributes● Rank your pages based on the value of the page● Fix JS errors immediately
![Page 21: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/21.jpg)
What else… some ideas.
● using conventional commit messages to determine hotspots in the code base: e.g. many fixes in one file makes it to a hotspot
● one can use this hotspot information to automatically make code reviews obligatory for risky changes
● … there is more to come
![Page 22: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/22.jpg)
Questions? Answers!
![Page 23: Do we need a bigger dev data culture](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a64d2627f8b9a4d0b8b49f5/html5/thumbnails/23.jpg)
Image Sources and Links
https://flic.kr/p/spw5mUhttp://devopsreactions.tumblr.com/post/147637425073/looking-for-a-bug-in-an-unknown-projecthttp://devopsreactions.tumblr.com/post/147386843655/high-traffic-hits-the-operations-teamhttp://www.iacquire.com/blog/speed-matters-improve-usability-with-google-analytics-site-speed-reports