Back to Best
-
Upload
brandon-hale -
Category
Documents
-
view
212 -
download
0
description
Transcript of Back to Best
Season of mists and mellow fruitfulness,
Close bosom-friend of the maturing sun;
Conspiring with him how to load and bless
With fruit the vines that round the thatch-eves run;
To bend with apples the moss'd cottage-trees,
And fill all fruit with ripeness to the core;
To swell the gourd, and plump the hazel shells
With a sweet kernel; to set budding more,
And still more, later flowers for the bees,
Until they think warm days will never cease,
For Summer has o'er-brimm'd their clammy cells.
Who hath not seen thee oft amid thy store?
Sometimes whoever seeks abroad may find
Thee sitting careless on a granary floor,
Thy hair soft-lifted by the winnowing wind
Or on a half-reap’d furrow sound asleep,
Drowsed with the fume of poppies, while thy hook
Spares the next swath and all its twinèd flowers:
And sometimes like a gleaner thou dost keep
Steady thy laden head across a brook;
Or by a cyder-press, with patient look,
Thou watchest the last oozings, hours by hours.
Where are the songs of Spring? Ay, where are they?
Think not of them, thou hast thy music too,—
While barrèd clouds bloom the soft-dying day
And touch the stubble-plains with rosy hue;
Then in a wailful choir the small gnats mourn
Among the river-sallows, borne aloft
Or sinking as the light wind lives or dies;
sinking
And full-grown lambs loud bleat from hilly bourn;
Hedge-crickets sing; and now with treble soft
The redbreast whistles from a garden-croft;
And gathering swallows twitter in the skies.
Everything You Wanted to Know
About Writing Async, Concurrent HTTP Apps
in Java
Agenda
Mostly this:
Agenda
And this:
Agenda
And this:
What Frog?
What Frog?
What Frog?
What Frog?
Requirements– parallel file Downloads– Parallel file parts– interrupt/pause/resume– Progress events– Checksums caching
First Association for “concurrent downloader”
Lucky day: Download manager written in java!
Let’s look if we can use it!
1.No traceable license2.No website or docs3.No traceable sources4. It’s an app, not a lib
Java.net.urlconnection1. Memory wasteful (buffering)2. Minimal API3. Blocking streams
What we’re looking for1. Async/non-blocking2. Event callbacks
What is IT going to take1. Reactor2. nio
Welcome to the reactor
– pattern for lightweight concurrency
– Event driven– Threads reuse– Uses non-blocking Io
Original pattern
http://www.dre.vanderbilt.edu/~schmidt/PDF/reactor-siemens.pdf
Guess the author by the diagram
http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf
In Java, Reactor
means NIO
Selector as a multiplexer
Java version - RegisteringSocketChannel channel= SocketChannel.open();
socketChannel.connect(new
InetSocketAddress("http://remote.com", 80));
...
Selector selector = Selector.open();
channel.configureBlocking(false);
SelectionKey k = channel.register(selector, SelectionKey.OP_READ);k.attach(handler);
Java version - Dispatcherwhile (!Thread.interrupted()) {
selector.select();Set selected = selector.selectedKeys();Iterator it = selected.iterator();while (it.hasNext()) SelectionKey k = (SelectionKey)(it.next(); ((Runnable)(k.attachment())).run(); selected.clear();
}
Handling reactor events is complex
–Need to maintain state–Buffering – assembling chunks
–Coordinating async events
Nio libraries–Most of them are servers
–Netty, grizzly, etc.–Apache Mina–Apache HTTP components asyncclient
–Ning http client
–Client and server nio library–Evolved from netty–Latest release October 2012
Nio libraries
Ning’s async http client
Here it is!
try (AsyncHttpClient asyncHttpClient = new AsyncHttpClient()) { ListenableFuture<Response> future = asyncHttpClient.prepareGet( "http://oss.jfrog.org/api/system/ping").execute( new AsyncCompletionHandler<Response>() { @Override public Response onCompleted(Response response) { System.out.println(response.getResponseBody()); return response; }
@Override public void onThrowable(Throwable t) { t.printStackTrace(); } }); Response response = future.get();}
HAC Concepts–Request producer–Response consumer
Choosing between ning and http asyncclient
"All problems in computer science can be solved by another level of indirection"
David Wheeler
public interface HttpProviderDownloadHandler {
void onResponseReceived(int statusCode, Map<String, List<String>> headers);
boolean onBytesReceived(ByteBuffer buf);
void onFailed(Throwable error);
void onCanceled();
void onCompleted();}
Head to head
Feature/Library Ning client Http Async ClientMaturity Good Very new (early 2014)Download cancelation
Easy With bugs
Progress hooks
Events not granular enough
Just use onByteReceived()
Documentation A bit sparse MinimalServer-side counterpart
None, client only
org.apache.httpcomponentshttpcore-nio
Performance?
http://blogs.atlassian.com/2013/07/http-client-performance-io/
Rfc2616: a universe of its own
Confused?
Just read some stackoverflow(and improve your rep as you go)
And that one for
discovering that range
header is lost on redirect
Question!
What should be content-length when using compression?
https://github.com/http2/http2-spec/issues/46
Question!
Why when redirected to CDN all the chunks start from zero?
HttpAsyncClientBuilder builder = HttpAsyncClients.custom();// add redirect strategy that copies "range" headers, if existbuilder.setRedirectStrategy(new DefaultRedirectStrategy() {
@Overridepublic HttpUriRequest getRedirect(HttpRequest request, HttpResponse
response, HttpContext context)HttpUriRequest redirectRequest = super.getRedirect(request,
response, context);// copy "Range" headers, if existHeader[] rangeHeaders = request.getHeaders(HttpHeaders.RANGE);
if (rangeHeaders != null) {for (Header header : rangeHeaders) {
redirectRequest.addHeader(header);}
}return redirectRequest;
}});
Question!
How many simultaneous connections should I open?
Question!
What’s wrong with the following code?
public static String encodeUrl(String urlStr) { URLEncoder.encode(urlStr, "UTF-8"); ...}
Decoded URLs cannot bere-encoded to the same form
http://example.com/?query=a&b==c
Cannot be decoded back after it was encoded:
http://example.com/?query=a%26b==c
Don’t use java.net.URLEncoder
“Utility class for HTML form encoding. This class contains static methods for converting a String to the application/x-www-form-urlencoded MIME format.For more information about HTML form encoding, consult the HTML specification.”
AHC Alternatives
Question!
How do I close a socket correctly?
How hard can it be to close a socket?
The art of socket closing
http://www.safaribooksonline.com/library/view/http-the-definitive/1565925092/ch04s07.html
Half-closed: no new customers
Never block in socket close()
The other side expects you to clean up nicely
It will give up on time outYou will wait (forever)
Remember?
Question!
How can I write file parts concurrently?
– Write to separate files, combine on finish
– Write to same file, seeking to the right position
private FileLock lock(FileChannel fileChannel) throws IOException { FileLock lock = fileChannel.tryLock(Long.MAX_VALUE - 1, 1, false); if (lock == null) { throw new OverlappingFileLockException(); } return lock;}
OS Level File Locking –Advisory exclusive
WTF?!
VM Level File Locking
VM Level File Locking–Prevent same VM threads writing to the file when we started closing it
–Closing sequence:–Release file locks–Close channels–Rename a file to it's final name (remove .part)
–Erase progress info
VM Level File LockingReentrantReadWriteLock.ReadLock writeToFileLock = rwl.readLock();ReentrantReadWriteLock.WriteLock closeFileLock = rwl.writeLock();
public void close() throws IOException { this.closeFileLock.lock();}
public int write(int partIndex, ByteBuffer buf) { if (!this.writeToFileLock.tryLock()) { throw new IllegalStateException("File is being closed"); } ...}
What’s next?
http/2– Mostly standardizing Google's
spdy– Header compression– multiplexing– Prioritization– Server push – On the way clear some stuff– E.g. compressed content length
Ease the load
Links!
RTFM: RFC 2616Ultimate book: HTTP: The Definitive Guide
AmazonSafari
Reactor patternDoug Lea on NIO
No, Thank you!
A Quick Tourof Logos
The Logical Appeal
So what exactly is logic? Who cares?Informally, logic is about saying things that make sense. You can think of it in that way if you like.
“It's pretty sunny today, so you should wear sunscreen.”
Formally, logic is the art of arguing – not like a fight or debate, but by using the information we already
know to draw new and useful conclusions.
“If it's sunny today, you should wear sunscreen.Indeed it is sunny today.
Therefore, you should wear sunscreen.”
“But wait. That just looked like the exact same thing you said before, you hack.”
Well, yes. But that's how an argument looks in standardform! You can break down any argument into this form;that makes it easier to think about.
If it's sunny today, you should wear sunscreen.
It is sunny today.___________________________________________
Therefore, you should wear sunscreen.
Premises/givens
Conclusion(Premises always come first, and the conclusion always comes last.)
How about a more complex argument?
1. This piece of fresh fruit is fuzzy.2. It also has seeds.
3. If a fruit is fuzzy, it's either a kiwi or a peach.4. Peaches have a pit; they don't have seeds.
5. So the fruit can't be a peach.6. So the fruit must be a kiwi.
Which of the above sentences is a conclusion?
How about a more complex argument?
1. This piece of fresh fruit is fuzzy.2. It also has seeds.
3. If a fruit is fuzzy, it's either a kiwi or a peach.4. Peaches have a pit; they don't have seeds.
5. So the fruit can't be a peach.6. So the fruit must be a kiwi.
Good logic lets us cobble together lots ofdifferent pieces of information, and tell from
them what's probably or definitely true.
But what counts as “good logic”?
That argument was good (made sense), because the conclusion followed from the premises. We'll see what this means in a moment.
Why don't we look at a bad argument?
But what counts as “good logic”?
Some people have fallen off cliffs and lived.
Therefore, if I jump off this cliff, I will definitely be fine.
“Come on.What couldpossibly gowrong?”
But what counts as “good logic”?
Some people have fallen off cliffs and lived.
Therefore, if I jump off this cliff, I will definitely be fine.
This argument is weak. Although the premise is true,it's easy to think of ways (very painful ways) that theconclusion could be false. The easiest way to spot badlogic is to do just that: try to think of another way out.
(Philosophers call these counterexamples).
But what counts as “good logic”?
Let's look at two kinds of arguments.1. Deductive reasoning: All interns can breathe fire. So Philip
can breathe fire.
Is there a piece of the puzzle missing?
But what counts as “good logic”?
Let's look at two kinds of arguments.1. Deductive reasoning: All interns can breathe fire. Philip is an
intern. So Philip can breathe fire.
Sometimes you may encounter “hidden” statementsand ideas, which the writer sneaks in but doesn'tsay outright.
But what counts as “good logic”?
Let's look at two kinds of arguments.1. Deductive reasoning:
All interns can breathe fire. Philip is an intern. So Philip can breathe fire.
Are the premises true?If so, then the conclusion's100% guaranteed true.No getting around it!
But what counts as “good logic”?
Let's look at two kinds of arguments.2. Inductive reasoning:
I touched a stove and it burned me. I did this fifty times, and the same thing happened. So
the next time I touch the stove, it will burn me.
Are the premises true?If so, then the conclusion'sprobably true. There mightstill be exceptions.
LIKE WHAT?
How is this useful to me?
Like Mr. Morgan said, the ability to make strong logical arguments will become more and more important later on in high school and college.
Pathos and ethos are still valuable! But your audience will be a lot better at questioning them. Logos is handy because, if you use it well, it can't really be disproved.
How is this useful to me?But even better is the superpower to spot weak logic.
Next time you watch TV or go online (with your parents' permission, of course), try to keep track of how many different arguments are being pitched to you by ads. How much info is given to you? How much is left out?
What time is it?Adven- wait, no. Activity time!
Pair off into groups of four. Each group will receive an exampleof a poor argument (these may be either inductive or deductive).
With your group, you will have 5 minutes to try to come up withone counterexample - one way in which the argumentcould be wrong, even if the premises are definitely true.Poke it full of holes!Also, choose a group representative to tell us your reasoning.
(It's OK to imagine unlikely or weird explanations;don't be afraid to think outside the box.Oddly enough, logic has very little to do with facts.)
Some examples:ARGUMENT: “I pulled an all-nighter studying for last week's bigtest, and I ended up with an A. Tiredness must make me smart!”
COUNTER: What if you got an A because you actually studied?Or maybe the test was going to be easy for you all along?
ARGUMENT: “If I play with Dad's power tools, he'll yell at me.But Dad is yelling at me for something. So I guess I must haveplayed with the power tools.”
COUNTER: What if he's yelling at you for a different reason:scratching the car, or hammer-throwing the cat onto the roof?
Season of mists and mellow fruitfulness,
Close bosom-friend of the maturing sun;
Conspiring with him how to load and bless
With fruit the vines that round the thatch-eves run;
To bend with apples the moss'd cottage-trees,
And fill all fruit with ripeness to the core;
To swell the gourd, and plump the hazel shells
With a sweet kernel; to set budding more,
And still more, later flowers for the bees,
Until they think warm days will never cease,
For Summer has o'er-brimm'd their clammy cells.
Who hath not seen thee oft amid thy store?
Sometimes whoever seeks abroad may find
Thee sitting careless on a granary floor,
Thy hair soft-lifted by the winnowing wind
Or on a half-reap’d furrow sound asleep,
Drowsed with the fume of poppies, while thy hook
Spares the next swath and all its twinèd flowers:
And sometimes like a gleaner thou dost keep
Steady thy laden head across a brook;
Or by a cyder-press, with patient look,
Thou watchest the last oozings, hours by hours.
Where are the songs of Spring? Ay, where are they?
Think not of them, thou hast thy music too,—
While barrèd clouds bloom the soft-dying day
And touch the stubble-plains with rosy hue;
Then in a wailful choir the small gnats mourn
Among the river-sallows, borne aloft
Or sinking as the light wind lives or dies;
sinking
And full-grown lambs loud bleat from hilly bourn;
Hedge-crickets sing; and now with treble soft
The redbreast whistles from a garden-croft;
And gathering swallows twitter in the skies.