Kdahlby 200908 Stldodn Linqinternals 090903222505 Phpapp01

LINQ Internals

Keith Dahlbyhttp://solutionizing.net/@dahlbyk

Who am I?

Iowa Native Iowa State University Cedar Rapids SharePoint Inetium Language Geek

Agenda

Terminology Enabling Technologies Query Expressions & Translation Standard Query Operators LINQ to Objects LINQ to IQueryable Beyond

Terminology

Language-INtegrated Query Query Expressions

o C# 3.0 & VB 9 Featureo Syntactic Sugar on Method Calls

LINQ Providero Standard Query Operatorso *LINQ / LINQ to *

Enabling Technologies

Anonymous Typeso Type Inference

Extension Methods Lambda Expressions

o Expression Trees

Anonymous Types

var entry = new { Title = "Dear Diary", DateTime.Now};

Console.WriteLine("{0:d}: {1}", entry.Now, entry.Title);

Dim entry = New With { _ .Title = "Dear Diary", _ DateTime.Now _}

Console.WriteLine("{0:d}: {1}", _ entry.Now, entry.Title)

var and Dim types are inferred!

Extension Methods

Must be defined in static class (VB Module) Appear in IntelliSense if namespace in using/Imports

First argument serves as instance variableImports System.Runtime.CompilerServices

Module StringExtensions <Extension()> _ Public Sub Print(ByVal aString As String) Console.WriteLine(aString) End SubEnd Module

Dim hello = "Hello from StringExtensions"hello.Print()

Extension Methods

C# supports this keywordstatic void ForEach<T>(this IEnumerable<T> source, Action<T> action){ foreach (T o in source) action(o);} Generic Delegatesdelegate void Action();delegate void Action<T>(T arg);delegate void Action<T1, T2>(T1 arg1, T2 arg2);delegate TResult Func<TResult>();delegate TResult Func<T, TResult>(T arg);delegate TResult Func<T1, T2, TResult>(T1 arg1, T2 arg2);

Lambda Expressions

Evolution of C# Delegatesdelegate string MyDelegate(int i);

static string MyMethod(int i) { … }MyDelegate del1 = new MyDelegate(MyMethod); // C# 1.0MyDelegate del2 = delegate(int i) { … }; // C# 2.0MyDelegate del3 = MyMethod; // C# 2.0MyDelegate del4 = (int i) => { … }; // C# 3.0Func<int, string> del5 = i => { … }; // C# 3.0

New in VB 9.0Dim del4 As MyDelegate = Function(i As Integer) …

Expression Trees

System.Linq.Expressions Abstract Syntax Tree representing code Assignable from lambda expression:Expression<Func<int, int>> inc = (a => a + 1); Compiles into:ParameterExpression CS$a;Expression<Func<int, int>> inc = Expression.Lambda<Func<int, int>>( Expression.Add( CS$a = Expression.Parameter(typeof(int), "a"), Expression.Constant(1, typeof(int))), new ParameterExpression[] { CS$a });

Query Expressions

Provide query keywords in C# & VBvar names = from s in Students where s.Age % 2 == 0 orderby s.Name descending select s.Name;

Translated into method calls:IEnumerable<string> names = Students.Where(s => s.Age % 2 == 0) .OrderByDescending(s => s.Name) .Select(s => s.Name);

Query Translation

Part of language specificationo Section 7.15.2 of C# 3.0 Language Specificationo Section 11.21 of Visual Basic 9.0 Language Specification

C# translation is rule-based:o “A query expression is processed by repeatedly applying

the following translations until no further reductions are possible.”

VB translation is sequential:o “Query operator translation occurs in the order in which

the query operators occur in the expression.”

C# Query Translation

Select and GroupBy clauses with continuationso from … into x … from x in ( from … ) …

Explicit range variable typeso from T x in e from x in ( e ) . Cast < T > ( )o join T x in e on k1 equals k2 join x in ( e ) . Cast < T > ( ) on

k1 equals k2

Degenerate query expressionso from x in e select x ( e ) . Select ( x => x )

From, let, where, join and orderby clauseso from, let and join are complicatedo from x in e where f … from x in ( e ) . Where ( x => f ) …o from x in e orderby k1 , k2 , … , kn … from x in ( e ) . OrderBy ( x =>

k1 ) . ThenBy ( x => k2 ) . … . ThenBy ( x =>

kn ) …

Select clauseso from x in e select v ( e ) . Select ( x => v )

GroupBy clauseso from x in e group x by k ( e ) . GroupBy ( x => k )o from x in e group v by k ( e ) . GroupBy ( x => k , x =>

Demo: Queries and Translation

Standard Query Operators: 51 Total

Aggregate Average Count LongCount

Max Min Sum Concat

AsEnumerable Cast ToArray ToDictionary

AsQueryable OfType ToList ToLookup

ElementAt First Last Single

ElementAtOrDefault

FirstOrDefault LastOrDefault SingleOrDefault

SequenceEqual GroupBy Join GroupJoin

DefaultIfEmpty Empty Repeat Range

Skip SkipWhile Take TakeWhile

Distinct Select SelectMany Where

Except Any All Contains

Intersect OrderBy ThenBy Reverse

Union OrderByDesc ThenByDesc

Standard Query Operators: 42 “Real” Operators

Max Min Sum Concat

ElementAtOrDefault

Standard Query Operators: 24 in Visual Basic 9

Max Min Sum Concat

ElementAtOrDefault

Standard Query Operators: 11 in C# 3

Max Min Sum Concat

ElementAtOrDefault

Standard Query Operators

Aggregation Operations Concatenation Operations Converting Data Types Element Operations Equality Operations Generation Operations Grouping Data Join Operations Partitioning Data Projection Operations Quantifier Operations Filtering Data Set Operations Sorting Data

Aggregations Operations

Max Min Sum Concat

ElementAtOrDefault

Concatenation Operations

Max Min Sum Concat

ElementAtOrDefault

Converting Data Types

Max Min Sum Concat

ElementAtOrDefault

Element Operations

Max Min Sum Concat

ElementAtOrDefault

Equality Operations

Max Min Sum Concat

ElementAtOrDefault

Generation Operations

Max Min Sum Concat

ElementAtOrDefault

Grouping Data

Max Min Sum Concat

ElementAtOrDefault

Join Operations

Max Min Sum Concat

ElementAtOrDefault

Partitioning Data

Max Min Sum Concat

ElementAtOrDefault

Projection Operations

Max Min Sum Concat

ElementAtOrDefault

Quantifier Operations

Max Min Sum Concat

ElementAtOrDefault

Filtering Data

Max Min Sum Concat

ElementAtOrDefault

Set Operations

Max Min Sum Concat

ElementAtOrDefault

Sorting Data

Max Min Sum Concat

ElementAtOrDefault

Closer Look: Join & GroupJoin

This query expression:from x1 in e1

join x2 in e2 on k1 equals k2 …

Translates into:from * in ( e1 ) . Join( e2 , x1 => k1 , x2 => k2 , (x1 , x2) => new { x1 , x2 })…

Join parameters:o this outer – The first sequence to join.o inner – The sequence to join to the first sequence.o outerKeySelector – Select key from first sequence. o innerKeySelector – Select key from second sequence.o resultSelector – Select result from two matching elements.

Closer Look: Join & GroupJoin

Use default equality comparero EqualityComparer<TKey>.Default

Composite keyso Type with good Equals() & GetHashcode()o Anonymous types good choice – required by some providers

Custom IEqualityComparer<TKey>o No query syntaxo Not alone

Operators With Comparer Overload

Max Min Sum Concat

ElementAtOrDefault

Closer Look: Join, Group Join and Left Outer Join

Join (no into)o “Correlates the elements of two sequences based on matching

keys.”o Func<TOuter, TInner, TResult> resultSelector

Group Join (with into)o “Correlates the elements of two sequences based on equality

of keys and groups the results.”o Hierarchical: One on left to many on righto Func<TOuter, IEnumerable<TInner>, TResult>

resultSelector Left Out Join

o Use DefaultIfEmpty() on Group Join result:from x1 in e1

join x2 in e2 on k1 equals k2 into jfrom xj in j . DefaultIfEmpty()…

Closer Look: SelectMany

Used to translate multiple from clauseso “Projects each element of a sequence to an IEnumerable<T>

and flattens the resulting sequences into one sequence.” This query expression:from x1 in e1

from x2 in e2

… Translates into:from * in ( e1 ) . SelectMany(

x1 => e2 , ( x1 , x2 ) => new { x1 , x2 } )… What is * ?

o Transparent Identifier = Intermediate Step

LINQ to Objects

Implementation of all Standard Query Operators System.Linq.Enumerable

o Extension methods on IEnumerable<T> Use C# Iterators Extensively

o yield return LINQBridge – .NET 2.0

o LINQ to Objectso Generic delegates (Action, Func)o ExtensionAttributeo http://code.google.com/p/linqbridge/

Manner of Execution

Immediateo Performed at the point where the query is declared.o All operators returning a non-enumerable result.

Deferredo Performed at the point where result is used.

• foreach• Immediate Operator

o Result depends on data source at point of use Deferred Streaming

o Consumes source data on demand Deferred Non-Streaming

o Caches all source data to perform operation

Immediate Execution

Max Min Sum Concat

ElementAtOrDefault

Deferred Streaming Execution

Max Min Sum Concat

ElementAtOrDefault

Deferred Non-Streaming Execution

Max Min Sum Concat

ElementAtOrDefault

LINQ to * via Objects

LINQ to XMLo System.Xml.Linqo All selectors return IEnumerable<>

LINQ to DataSeto System.Data.DataSetExtensionso EnumerableRowCollection<DataRow>

AsEnumerable(this DataTable source) o EnumerableRowCollectionExtensions provides some

operators LINQ to Legacy

o Take advantage of Cast<T>() and OfType<T>()o Query expression support, just like foreach:

from MyType obj in MyArrayList…

Tour of LINQ to XML and DataSet Internals

LINQ to IQueryable

Expression Trees from Standard Query Operators System.Linq.Queryable

o Extension methods on IQueryable<T>

public interface IQueryable : IEnumerable{ Type ElementType { get; } Expression Expression { get; } IQueryProvider Provider { get; }}

public interface IQueryable<T> : IEnumerable<T>, IQueryable, IEnumerable{ }

IQueryProvider

“Only” responsibility is processing expression trees

public interface IQueryProvider{ IQueryable CreateQuery(Expression expression); IQueryable<TElement> CreateQuery<TElement>(Expression expression);

object Execute(Expression expression); TResult Execute<TResult>(Expression expression);}

Queryable Operators

public static TSource First<TSource>( this IQueryable<TSource> source){ return source.Provider.Execute<TSource>( Expression.Call(null, ((MethodInfo) MethodBase.GetCurrentMethod()) .MakeGenericMethod(new Type[]

{ typeof(TSource) }), new Expression[] { source.Expression } ) );}

Tour of LINQ to SQL Internals

Beyond?

IEnumerable and IQueryable are not special Methods can come from anywhere Parameter types are flexible

o As long as the compiler can infer everything…o Who ever said LINQ predicates need to be Boolean-value

New provider modelso LINQ to Eventso Lazy LINQ (coming soon)

Resources

C# in Depth by Jon Skeet (Manning) Bart De Smet – Microsoft Language Geek

o http://community.bartdesmet.net/blogs/bart/

Meo http://solutionizing.net/o keith@ ---^o @dahlbyk

Kdahlby 200908 Stldodn Linqinternals 090903222505 Phpapp01

Documents

Transcript of Kdahlby 200908 Stldodn Linqinternals 090903222505 Phpapp01

Congress 091005124402 Phpapp01 091014131158 Phpapp01

The Triton 200908

200908 2019-2020 Final EHS HS Annual Report

Governmentassignmentkennychrisaustin 091005142914 Phpapp01 091007142214 Phpapp01

Icttosupportacreativecurriculuminyears1and2 101006034102-phpapp01-101006115049-phpapp01

STLDODN - Agile Testing in a Waterfall World

GettingStartredwithSPSS 17 PartI 200908

Lirecruiter 1337849509068 Phpapp01 120524035218 Phpapp01

Owtv 13336434806874 Phpapp01 120405113501 Phpapp01

200908.Ej Lc Wv.Presentation Briefing Head Office

Kelman Transfix GEA-17280D-E 200908 R002

WIRELESS KIT 200908 - m.media-amazon.com

Cashflowmvit 13117382206982-phpapp01-110726230250-phpapp01

Uspsocialmediaseriespharma4 1330710013868 Phpapp01 120302114453 Phpapp01

LINQ Internals - STLDODN

Supply Demand Chain Executive (Sdce) 200908

Devel::NYTProf v3 - 200908 (OUTDATED, see 201008)

200908 ASAP OptimalPopulationReport

Totaldefenseproductinformation 1327447773974 Phpapp01 120124173019 Phpapp01

OData feed for that - stldodn