Programming on Windows 8.1: The New Stream and Storage Paradigm (Raffaele Rialdi)

Premium community conference on Microsoft technologies itcampro@ itcamp14#

The new stream and storage paradigm

twitter: @raffaeler

email: [email protected]

blog: http://www.iamraf.net

Programming on Windows 8.1:


Huge thanks to our sponsors & partners!


• ActivateInstance API to create objects

• Native types

• IVector<T> and IMap<T, K>

• HRESULT for errors

.NET Framework Projection magics

• Familiar new keyword

• Familiar BCL types

• List<T> and Dictionary<T, K>

• Exceptions

WinRT .NET projection

But projections and mappings can’t fix all the frictions


• The need to enforce the async paradigm

– “fast and fluid” means a better UX

–Blocking apps are crappy!

• Solutions:

– async/await

–Promise pattern

The asynchronous problem


• async / await language feature is easy

• Promise is a ‘continuation’

Async quick recap

var content = await FileIO.ReadTextAsync(storageFile);

lblResult.Text = result;

var content = File.ReadAllText(path);lblResult.Text = result;

Callback

Sync

Async

create_task(FileIO::ReadTextAsync(storageFile)).then( [ = ] (String^ content) {lblResult->Text = result;

}

C++ Promise


• The need to enforce a sandbox

–Protecting the system and user’s resources from the Apps

• A sandbox is substantially a process with low privileges

–App process is created with an very poor token

– Leverage the Integrity Levels

–A high number of APIs cannot be used

• sockets, System.IO, Streams, http api, …

The security problem


• System.IO

– Let you browse/navigate the disk

– Let you access absolute path

– Use the old-school synchronous pattern

• Storage API

– Must restrict the access to few well-known locations

• Can access arbitrary locations picked from the user

– Must never block the main thread

• Security and Async are two requirements not allowing mapping or conversion

The I/O APIs have both those problems


• Application Package– The place where EXE and DLL lives in

• Using MEF or an IoC? This is your code repository

– Xaml Uri: "ms-appx:///"

• If the manifest specified special folders …… and the Store certification approved it

– KnownFolder contains the special folders• Documents, Pictures, Music, etc.

– Note: Documents capability is hidden in VS and its usage is permitted only to company accounts

What can an app access to?

Windows.ApplicationModel.Package.Current.InstalledLocation


• ApplicationData.Current.LocalFolder– Local assets, no storage limits

– Xaml Uri: "appdata:///local/"

• ApplicationData.Current.RoamingFolder– Synced content, limited in size

• ApplicationData.RoamingStorageQuota (100K)

– Xaml Uri: "ms-appdata:///roaming/"

• ApplicationData.Current.TempFolder– Use only for short duration operations

– Xaml Uri: "ms-appdata:///temp/"

Application-specific folders


• Files shipped with the package–Mark them as 'content' in Visual Studio

– "Copy Always" ensure fresh updated content

– From XAML use "ms-appx:///" or "/"

– From code use • Windows.ApplicationModel.Package.Current.

InstalledLocation

• var img = new Uri("ms-appx:///Assets/Logo.png");– Uri as only absolute in WinRT

Resources

<Image Source="/Assets/Logo.png" Height="100"/><Image Source="ms-appx:///Assets/Logo.png" Height="100"/>


DEMO:A LAP AROUND STORAGE API


• There are still missing methods in some classes of the WinRT API

• .NET users can use Extension Methods in the shared code to fill the gap– Add a method which is identical to the one

available in Win8.1

• Trick! Extension methods do not prevail over real ones– Win8.1 will execute the real WinRT one

– WP8.1 will execute the extension method

Windows Phone 8.1 API differences


• Do you remember the security constraints?

– Poor token, low integrity level, restricted APIs

• Sometimes we need to bypass the limitations

– Accessing a file outside the ones owned by the App

• Pickers use the Runtime Broker to bypass the limitation

– Accesses the resources for the requesting App

– User is always in control of the Broker requests

Pickers


Pickers and the Runtime Security Broker

Kernel services

WinRT

RuntimeSecurity Broker

COM / Win32 filtered Complete COM / Win32

Component

Deviceaccess

PickerHost

…

Component

Broker UIProcess

AppContainerProcess

Restricted tokenLow Integrity Level

Standard tokenMedium Integrity Level


WORKING WITH STREAMS


• Streams can block the client code

–Until you read/write, you will never know

• Working in chunks makes sense

– Files, network, … typically send/receive chunks

• Buffering limits the use of async to loading or writing the buffer

• WinRT use a low level buffering concept

– IBuffer interface representing byte arrays

The need of buffering


• It's made only of Capacity and Length– Creation via Buffer class (no read/write methods)– Write access is possible via

• WindowsRuntimeBuffer.Create• Low level access via COM interface

– IBufferByteAccess (C++ only)

– Conversions/mappings by:

IBuffer and friends

WinRT Type .NET Type .NET WinRT WinRT .NET

IBuffer Byte[] AsBuffer, CopyTo ToArray, CopyTo

IBuffer Byte N/A GetByte

IBufferStreamMemoryStream

AsStream,GetWindowsRuntimeBuffer AsStream

WindowsRuntimeBufferExtensions


• DataReader can read IBuffer

• IInputStream and IOutputStream– Are backed up by IBuffer based buffers

IBuffer is used from other APIs

byte[] blob = Utiltities.CreateSampleBlob();byte[] blob2 = new byte[blob.Length];IBuffer buffer = blob.AsBuffer();using (DataReader reader = DataReader.FromBuffer(buffer)){

reader.ReadBytes(blob2);}Debug.Assert(Utiltities.IsSameDataAndDifferentReference(blob, blob2));

using (DataReader reader = DataReader.FromBuffer(buffer)){

buffer2 = reader.ReadBuffer((uint)blob.Length);blob2 = buffer2.ToArray();

}Debug.Assert(!buffer.IsSameData(buffer2));

WindowsRuntimeBufferExtensions


Streams hierarchy

IInputStreamReferenceIRandomAccessStream

Reference

IAsyncOperationWithProgress<IBuffer, uint>ReadAsync(IBuffer buffer,

uint count,InputStreamOptions options);

IAsyncOperationWithProgress<uint, uint> WriteAsync(IBuffer buffer);

IAsyncOperation<bool> FlushAsync();

string ContentType {get; }

IRandomAccessStream CloneStream();IInputStream GetInputStreamAt(

ulong position);IOutputStream GetOutputStreamAt(

ulong position);void Seek([In] ulong position);

bool CanRead { get; }bool CanWrite { get; }ulong Position { get; }ulong Size { get; set; }

Sequential access interfaces

IInputStream IOutputStream

IRandomAccessStream

IRandomAccessStreamWithContentType

IClosable(IDisposable)


• Maintain internally an IBuffer• LoadAsync load the buffer• ReadXYZ methods hit the buffer, not the stream!

– They can be safely synchronous

DataReader puts IBuffer and streams together

Stream

I/OLatency

0 Size

LoadAsync LoadAsync LoadAsync

IBuffer

DataReader

ReadBooleanReadByte

ReadBytes…

var file = await Package.Current.InstalledLocation.GetFileAsync(@"Assets\Logo.png");

using (var stream = await file.OpenReadAsync()){

using (var reader = new DataReader(stream)){

await reader.LoadAsync((uint)stream.Size);Debug.WriteLine("0x" +

reader.ReadByte().ToString("x2"));} // stream is disposed here ...// ... unless reader.DetachStream() is not called

}


• Writes are against the IBuffer

• The stream is written only at StoreAsync time

• FlushAsync can be used to flush the underlying stream

DataWriter use the same schema

Stream

I/OLatency

0 Size

StoreAsync StoreAsync StoreAsync

IBuffer

DataWriter

WriteBooleanWriteByte

WriteBytes…

// ... using (var writer = new DataWriter(stream)){

writer.WriteInt32(1);written = await writer.StoreAsync();writer.DetachStream();

}// ...


WinRT Type .NET Type .NET WinRT WinRT .NET

IInputStream Stream AsInputStream AsStreamForRead

IOutputStream Stream AsOutputStream AsStreamForWrite

IRandomAccessStream Stream AsRandomAccessStream AsStream

Converting streams between .NET and WinRTbyte[] blob = Utiltities.CreateSampleBlob();byte[] blob2 = new byte[blob.Length];

using (var memStream = new MemoryStream()){

await memStream.WriteAsync(blob, 0, blob.Length);memStream.Seek(0, System.IO.SeekOrigin.Begin);

using (var reader = new DataReader(memStream.AsInputStream())){

await reader.LoadAsync((uint)blob.Length);reader.ReadBytes(blob2);

}}Debug.Assert(Utiltities.IsSameDataAndDifferentReference(blob, blob2));


TIPS


• RandomAccessStream static class expose helper methods to copy a stream

–CopyAsync, CopyAndCloseAsync

• C++ will take advantage of async/await

– they makes a huge difference!

• IBufferByteAccess

Useful tips to remember

Microsoft::WRL::ComPtr<Windows::Storage::Streams::IBufferByteAccess>GetByteBuffer(Windows::Storage::Streams::IBuffer^ buffer)

{Microsoft::WRL::ComPtr<Windows::Storage::Streams::IBufferByteAccess> ibba;Microsoft::WRL::ComPtr<IUnknown> unknown(reinterpret_cast<IUnknown*>(buffer));unknown.As<Windows::Storage::Streams::IBufferByteAccess>(&ibba);return ibba;

}

HRESULT Buffer([out] byte** value);


Q & A

Programming on Windows 8.1: The New Stream and Storage Paradigm (Raffaele Rialdi)

Technology

Transcript of Programming on Windows 8.1: The New Stream and Storage Paradigm (Raffaele Rialdi)