SDN MAGAZINE KNOW- LEDGE#133kunstmatige intelligentie en machine learning een grote rol. Het feit...

ISS

N: 2211-6486

ONDERWERPEN:

Microsoft Cognitive Services

Eight Practices ForContainerized Delivery on the Microsoft stack

Cloning FDDataSet Cursors

Een simpel systeem voor beeldherkenning met

Microsoft Computer Vision API en Solr

Nummer 133 Oktober 2017 SDN Magazine verschijnt elk kwartaal en is een uitgave van Software Development Network

SDN Magazine is hét lijfblad van en voor programmeurs, met artikelen over ontwikkelen voor Windows, Web, Mobile of Cloud. Of u nu ontwikkelt voor .NET, Delphi of iets anders, als lid van het SDN weet u alles.

SDN MAGAZINEKNOW-LEDGE#133

IN DIT NUMMER BIJDRAGEN VAN O.A.:

Cary Jensen

Cornell Knulst

Sander Hoogendoorn

André Obelink

Roel Hans Bethlehem

magazine voor software development 3

ColofonUitgaveSoftware Development NetworkVijfentwintigste jaargangNo. 133 • oktober 2017

Bestuur van SDNMarcel Meijer, voorzitterRob Suurland, penningmeesterRemi Caron, secretaris

RedactieMarcel Meijer ([email protected])

Aan dit magazine werd meegewerkt doorRoel Hans Bethlehem, Bob Swart, Maarten van Stam, Arjen Bos, Fanie Reynders, Rob Suurland, Remi Caron, Marcel Meijer en natuurlijk alle auteurs!

ListingsZie de website www.sdn.nl voor eventuele source files uit deze uitgave.

ContactSoftware Development NetworkPostbus 506 7100 AM WinterswijkTel. (085) 21 01 [email protected]

Vormgeving en opmaakReclamebureau Bij Dageraad Winterswijk www.bijdageraad.nl

©2017 Alle rechten voorbehouden. Niets uit deze uitgave mag worden overgenomen op welke wijze dan ook zonder voorafgaande schrifte-lijke toestemming van SDN. Tenzij anders vermeld zijn artikelen op persoonlijke titel geschreven en verwoorden zij dus niet noodzake-lijkerwijs de mening van het bestuur en/of de redactie. Alle in dit maga-zine genoemde handelsmerken zijn het eigendom van hun respectieve-lijke eigenaren.

AdverteerdersAchmea . . . . . . . . . . . . . . . . . . . . . 2Microsoft . . . . . . . . . . . . . . . . . . . 28

AdverterenInformatie over adverteren en de advertentietarieven kunt uvinden op www.sdn.nl onder de rubriek Magazine.

voorwoord


Over Marcel Meijer:

Marcel Meijer is Freelance (Enterprise) Architect, Teamlead, CTO, CTO Advisor of interim manager. Daarnaast is Marcel voorzitter, bestuurslid, event organisator en eindredacteur bij de SDN. Sinds 2010 mag hij zich MVP noemen.

Beste SDN Magazine lezer,

Deze keer een SDN magazine met artikelen over Microsoft Cognitive Services. Er zijn al verschillende voorbeelden getoond op Microsoft, Google, Apple en Amazon events. Ieder groot IT technologie bedrijf is inmiddels wel aan het investeren in Artificial Intelligence en Machine Learning. En dit is nog maar het begin. In de toekomst zal dit nog veel verder uitgewerkt gaan worden, van verder gaande automatisering van vele arbeidsprocessen tot volledig autonoom rijdende auto’s. Bij het eerste hebben we het dan niet perse over de volgende iteratie van het automatiseren van het industriële proces, maar van processen waarbij nu nog veel mensenwerk nodig is. Het aanvragen van vergunningen of afsluiten van contracten. Mogelijk zelfs in combinatie met Blockchain technology en andere moderne tools.

De snelheid waarmee deze technologieën groeien en uitgewerkt worden, is soms verbazingwekkend en soms zelf een beetje beangstigend. Wat betekent dit voor de arbeidsmarkt als veel werk geautomatiseerd wordt? Wat betekent dat voor de regelgeving? Wat betekent dat voor de regeldruk? Hoe voorkomen we doom scenario’s zoals we zien in films als ‘The Terminator’, ‘I, Robot’, ‘Ex Machina’ etc. Dit zal allemaal wel loslopen, tenslotte is de begeleider die voor de automobiel op straat liep ook verdwenen, genieten we nog elke dag van de industriële automatisering en de verdergaande informatisering. Wel zullen we goed moeten opletten op onze privacy en de onderkant van de beroepsbevolking.

Naast artikelen over Artificial Intelligence in het bijzonder Cognitieve Services door Andre en Roel Hans, hebben we ook artikelen over Containers in combinatie met de Microsoft Stack, Project managers en Cloning datasets door Cornell, Sander en Cary. Kortom weer boeiende artikelen met echte handson informatie, die je nu en vandaag al kunt inzetten voor je lopende projecten.

Hou ook onze SDNCast uitzendingen in de gaten voor meer up-to-date informatie en informatie over de komende events. Ben je al geabonneerd op ons kanaal (elke week op donderdag na 20:00 uur)? Ga naar www.sdncast.nl voor meer info!Rest mij om namens alle medewerkers van de SDN veel leesplezier en tot ons volgende event of uitzending van de SDN Cast. Wil je schrijven voor ons magazine of spreken op onze events of meedoen aan onze cast meldt je dan op [email protected].

Groeten,Marcel MeijerEindredacteur Magazine en Voorzitter SDN

AGENDA 2017 / 2018

12-17 November Visual Studio Live!. Orlando

24-26 Oktober SharePoint Unite Haarlem

21 April 2018 Global Azure Bootcamp

15 Juni 2018 SDN event 2

6-9 November Web SummitLissabon, Portugal

23 Maart 2018 SDN event 1 + ALV

22-24 Mei 2018 Techorama België

14 December 2018 SDN event 3


SOFTWARE DEVELOPMENT NETWORK






SOFTWARE DEVELOPMENT NETWORK BEELDMERK

FULL-COLOUR

SOFTWARE DEVELOPMENT NETWORK DIVISIES

CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n

SOFTWARE DEVELOPMENT NETWORK SOFTWARE DEVELOPMENT NETWORK


4

Inhoud

03 Voorwoord Marcel Meijer

05 Inhoudsopgave

06 Microsoft Cognitive Services André Obelink

11 Eight Practices For Containerized Delivery on the Microsoft stack Cornell Knulst

16 Cloning FDDataSet Cursors Cary Jensen

22 Een simpel systeem voor beeldherkenning met Microsoft Computer Vision API en Solr Roel Hans Bethlehem

27 The Left-handed project magager Sander Hoogendoorn









FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



Serverless integration: LOgIC AppS eN FuNCTIONS

Steef-Jan Wiggers

6

MICROSOFT

Cognitive Services

André Obelink

Op het eerste gezicht lijkt dit allemaal best eenvoudig te realiseren. Je koppelt immers je agenda aan Cortana of een of andere culinaire website biedt zijn API aan. Maar wanneer je een stap verder kijkt en je gaat afvragen hoe Alexa of Siri nu precies weet wat jij vraagt dan wordt het wel ingewikkeld. Je hebt verschillende stemmen, verschil-lende intonaties, verschillende emoties en ook nog eens ontelbare manieren waarop je een vraag kunt formuleren. Bij de verwerking en interpretatie van deze informatie komt duidelijk kunstmatige intel-ligentie in beeld. Het is bijzonder interessant te weten hoe dit onder de motorkap allemaal werkt, maar het is minstens zo interessant en nuttig te weten hoe je deze technologie in je eigen software zou kun-nen toepassen.

In dit artikel stel ik jullie voor aan een aantal cognitieve diensten die Microsoft aanbiedt aan softwareontwikkelaars. De Microsoft Cogni-tive Services zijn een set van API’s, SDK’s en cloud services waar-mee je jouw applicaties een stuk intelligenter kunt maken. Je kunt hierbij denken aan gezichts- of voorwerpherkenning in foto’s of zelfs het herkennen van gezichten in foto’s en bepalen welke emotie dit gezicht uitstraalt. Daarnaast zijn er services ten behoeve van taal en spraakherkenning. Op de achtergrond speelt, zoals eerder gezegd, kunstmatige intelligentie en machine learning een grote rol. Het feit dat veel services slimmer worden naarmate ze meer en meer gebruikt worden, geven deze diensten nog meer meerwaarde dan ze initieel al hebben.

Microsoft maakt met Cognitive Services kunstmatige intelligentie dus beschikbaar voor iedereen. Volgens hen is dit nu mogelijk doordat drie elementen nu samenkomen: de computerkracht die Cloud com-puting biedt, de ontwikkeling van algoritmes en machine learning en als laatste de beschikbaarheid van grote hoeveelheden data. Hier kan men zich ook wel wat bij voorstellen.De cognitive services zijn beschikbaar als een API die je vanuit je applicatie kunt aanroepen. Per API die je wilt gebruiken moet je zoge-naamde ‘secret keys’ hebben. De meeste API’s kennen een probeer-versie die je toestaan om een bepaald aantal calls per maand te doen. Het is daarom verstandig om jouw API keys voor jezelf te houden. Daarnaast zijn sommige API’s is de probeerversie ook wat beperkt qua mogelijkheden. Om serieus met deze services aan de slag te gaan, moet je over een Microsoft-account en een Azure abonnement beschikken. Voor alleen wat testen hoef je niet een Azure abonnement te hebben.

Zodra de betaalde versies van de API’s beschikbaar komen, wordt dit gezien als een extra dienst in je Azure portal. Deze API-abonnementen hangen dus aan jouw Azure subscription. Je kunt hier instellen dat je gebruik wilt maken van de probeer- of betaalde versie, je kunt het aantal calls zien en vanuit hier heb je natuurlijk ook toegang tot je secret keys. In bovenstaande schermafbeelding zie je een voorbeeld van een Computer Vision API die ik, naast een Face API, heb aange-maakt onder de naam MarYorComputerVisionApi.

API-deelgebIedenDe beschikbare API’s zijn grofweg onder te verdelen in een vijftal deel-gebieden: Vision, Speech, Language, Knowledge en Search.• VisionGeavanceerde beeldverwerkingsalgoritmen helpen je door informatie te retourneren over gezichten, afbeeldingen en emoties. Voorbeelden zijn: Computer Vision API, Emotion API, Video API of de reeds ee-rder genoemde Face API. De Face API biedt als functionaliteit dat het gezichten in foto’s kan herkennen en vaststellen of twee gezichten tot dezelfde persoon behoren. De Computer Vision API werken we later in dit artikel uit als codevoorbeeld.• SpeechMet de Speech API’s kun je gesproken taal in je software verwerken. Voorbeelden zijn: Custom Speech Service, Translator Speech API, Bing Speech API of Speaker Recognition API. Met de laatste API kun je bijvoorbeeld automatisch een gebruiker herkennen op basis van zijn of haar stem.

Kunstmatige intelligentie is absoluut niet iets van de afgelopen jaren, maar staat momenteel wel sterk in de belangstelling. Het is niet voor niets dat grote bedrijven zoals Google, Amazon, Apple en Microsoft nu veel tijd, geld en energie steken in allerlei ontwikkelingen rondom Artificial Intel-ligence (AI). Denk alleen maar eens aan alle spraakassistenten die onze huiskamer betreden. Google Assistent, Siri, Alexa en Cortana maken al deel uit van ontzettend veel huishoudens. Het ophoesten van een recept van een appeltaart, het voorlezen van het weerbericht, het aandoen van verlichting, je waarschuwen dat je naar de tandarts moet… allerlei handige handelingen die je met je stem kan besturen.

CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL









FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



F I G U R E 1








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



8

| | | | | | | | | | | | | | | | | | | | | | |Serverless integration:

LOgIC AppS eN FuNCTIONS

nn

nn

nn

nn

nn

nn

nn








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



8

| | | | | | | | | | | | | | | | | | | | | | |MICROSOFT

Cognitive Services

nn

nn

nn

nn

nn

nn

nn

• LanguageMet deze set van API’s kun je natuurlijke taal verwerken, bepalen welke onderwerpen worden besprokenen en eventueel het senti-ment van de tekst. Hiermee kun je dus ook begrijpen wat gebruikers bijvoorbeeld met een vraag bedoelen. Voorbeelden zijn Language Understanding Intelligent Service (LUIS), Bing Spell Check API, Text Analytics API of de Translator Tekst API. Je zou LUIS kunnen gebruiken om een language model te maken die begrijpt of je de lamp boven de eettafel aan/of uit wilt doen of jouw bureaulamp of de lamp in de keuken.• KnowledgeVerwerk complexe informatie en data om zodoende intelligente aan-bevelingen te doen of om bijvoorbeeld semantisch te kunnen zoeken. Enkele API’s die tot deze groep behoren zijn: Recommendations API, Knowledge Exploration Service, Entity Linking Intelligence Service API, Academic Knowledge API, QnA Maker API en de Custom Deci-sion Service.• SearchGebruik Microsofts zoekmachine Bing binnen je eigen applicaties. Voorbeelden zijn: Bing Autosuggest API, Bing Image Search API, Bing News Search API, Bing Video Search API, Bing Web Search API, Bing Custom Search en Bing Entity Search API.

Nu je overzicht hebt gekregen van een groot aantal API’s die Micro-soft nu beschikbaar heeft gemaakt, wordt het de hoogste tijd om eens te kijken hoe je ze binnen je software aanroept. De verschillende Cognitive Services zijn erg divers, dus ook de toepassing ervan. Toch komen ze qua gebruik en implementatie aardig overeen. We gaan nu een voorbeeld uitwerken in C# die gebruik maakt van de Computer Vision API, maar dit had eigenlijk dus elke andere API kunnen zijn.

de ComPuter VIsIon API AAnroePenZorg dat je een Computer Vision API hebt aangemaakt en in het bezit bent van een secret key. Een logische stap daarna is om de documentatie van de API te bekijken. Hier worden alle operaties en mogelijke parameters uitgelegd. Ook heb je vanuit hier toegang tot de API testing console, waarmee je de API eenvoudig online al kunt aanroepen. Ik heb een link naar deze pagina opgenomen aan het einde van dit artikel. De Computer Vision API bevat (momenteel) de volgende operaties:• AnalyzeImage-POST• DescribeImage-POST• GetHandwrittenTextOperationResult-GET• GetThumbnail-POST• ListDomainSpecificModels-GET• OCR-POST• RecognizeDomainSpecificContent-POST• RecognizeHandwrittenTekst-POST• TagImage-POST

Elke operatie kent zijn eigen eindpunt met eventueel een aantal parameters. De opbouw van zo’n eindunt, de zogenaamde Request URI, gaat eigenlijk altijd via een standaard stramien:

https://[location].api.cognitive.microsoft.com/[api-name]/[version]/[operation]?[parameters]

In dit template kun je de volgende blokken onderscheiden:• location: de locatie waar je API wilt aanroepen, bijvoorbeeld ‘westus’ of ‘westeurope’• api-name: de naam van de API. In het geval van de Computer Vision Api is dat ‘vision’• version: de versie van de API. Nu is dat doorgaans altijd ‘v1.0’ • operation: de operatie die je wilt gebruiken, bijvoorbeeld ‘analyze’ of ‘describe’• parameters: afhankelijk van de operatie kan men ook een of meerdere parameters meegeven. In het geval van de operatie ‘analyze’ kun je hierbij denken aan dat je een lijst met categorieën en/of de hoofdkleur van een afbeelding terug wilt krijgen.

In ons voorbeeld waarbij we de Computer Vision API aan willen roepen kan het eindpunt of Uri er als volgt uit zien:

https://westeurope.api.cognitive.microsoft.com/vision/v1.0/analyze?visualFeatures=Categories

We maken een simpele applicatie waarmee we een afbeelding op de lokale PC kunnen selecteren, deze vervolgens aanbieden aan de API en het resultaat tonen. De userinterface stel niets voor, een knop (selectImageFilePathButton) om een bestand te selecteren en twee tekstvakken. In de eerste (imageFilePathTextBox) wordt de geselect-eerde bestandsnaam getoond en in het tweede (jsonResultTextBox) het resultaat na de aanroep van de API.

Laten we beginnen met code die wel noodzakelijk is, maar niet direct gerelateerd is aan het aanroepen van de API.

private async void selectImageFilePathButton_Click(object

sender, EventArgs e)

{

var imageFilePath = GetImageFilePath();

imageFilePathTextBox.Text = imageFilePath;

F I G U R E 2








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



10

“tags”: [

{ “name”: “water”, “confidence”: 0.99997103214263916 },

{ “name”: “outdoor”, “confidence”: 0.99797040224075317 },

{ “name”: “sky”, “confidence”: 0.99212568998336792 },

{ “name”: “shore”, “confidence”: 0.764807403087616 },

{ “name”: “nature”, “confidence”: 0.75505912303924561 },

{ “name”: “ocean”, “confidence”: 0.65932834148406982 }

sAmenVAttendIn dit artikel heb ik je laten kennis maken met Microsoft Cognitive Services en één API in het bijzonder: de Computer Vision API. De functionaliteit die deze Cognitive Services aanbieden zijn zeer geschikt om je applicaties slimmer te maken en de implementatie van de ben-odigde logica vergt weinig code. De mogelijkheden en toepassingen zijn eindeloos. Ik nodig je van harte uit om hiermee aan de slag te gaan!

gerelAteerde lInks• Codevoorbeeldhttps://github.com/obelink/MicrosoftCognitiveServicesComputerVi-sionDemo

• Overizchtallecognitiveserviceshttps://azure.microsoft.com/nl-nl/services/cognitive-services

• ComputerVisionAPIdocumentatiehttps://azure.microsoft.com/nl-nl/services/cognitive-services/com-puter-vision

| | | | | | | | | | | | | | | | | | | | | | |MICROSOFT

Cognitive Services

nn

nn

nn

nn

nn

nn

nn

Met relatief weinig code kun je de API vanuit C# of Visual Basic .NET aanroepen en het resultaat bekijken. Maar welk resultaat krijg je nu feitelijk terug? Laten we de proef op de som nemen.

de ComPuter VIsIon API In ACtIeWanneer we onze applicatie gaan testen, kunnen we dus vaststellen hoe slim de Computer Vision API Analyze methode nu eigenlijk is. Ik heb een groot aantal foto’s getest en ik ben echt onder de indruk van de kwaliteit van de geretourneerde data. Ik schrijf dit artikel vanuit mijn vakantieverblijf in Florida en heb gisteren onderstaande foto gemaakt:

We hebben aangegeven dat we een drietal informatieblokken gere-tourneerd willen hebben: Categories, Tags en Description. We krijgen echter altijd ook een sectie Captions terug. Laten we met de laatste beginnen:

“captions”: [

{

“text”: “a couple of people that are standing in front of

a body of water”,

“confidence”: 0.95049971721239168

}

De API is 95% zeker over het resultaat en we moeten hem nageven, de omschrijving klopt ook erg goed. Ook de sleutelwoorden zijn bijna allemaal relevant, hoewel er ook een aantal via associatie zijn toege-voegd en niet direct zijn herkend.

“description”: {

“tags”: [

“water”, “outdoor”, “ocean”, “man”, “beach”, “body”,

“standing”, “front”, “bird”, “pier”, “people”, “cou-

ple”,

“board”, “boat”, “large”, “umbrella”, “flying”, “sur-

fing”,

“rocky”, “dock”, “walking”, “group”, “white”, “riding”

],

En als laatste wil ik nog inzoomen op de Tags-sectie. Deze Tags zijn eigenlijk allemaal relevant.

AndrÉ obelInk

André Obelink is senior software engineer en eigenaar van MarYor | software & consultancy B.V. Hij is al jaren actief in de nationale- en internationale developer community als spreker en auteur van artikelen en boeken. Daarnaast is hij één van de mensen achter VBcentral.nl. Voor zijn inzet en bijdragen aan de developers community is hij vanaf 2006 elf jaar op rij onderscheiden met een Microsoft MVP award. Zijn blog vind je op: http://www.obelink.com

CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL

F I G U R E 3


EIGHT PRACTICES FOR

Containerized Delivery on the Microsoft stack

Cornell Knulst

During the past year I supported several clients in their journey toward Containerized Delivery on the Microsoft stack. In this article I’d like to share eight practices I learned while practicing Containerized Delivery on the Microsoft stack using Docker, both in a Greenfield and in a Brownfield situation.

PrACtICe 1: smAll, reusAble ImAge lAyersOnce you start containerizing.NET workloads, you need to decide how to modularize your container images. A good starting point is to review your architecture and determine which parts of your application landscape you need to selectively scale or release independently. In fact, each container image you create should be self-contained and must be able to run on its own. There is also another important aspect you have to think about: container image layering.

As you may or may not know, container images are the blueprint for your containers. Images consist of image layers. Each image layer is created during the Docker build process as the resulting artifact of a set of instructions (e.g., creating a directory, enabling Windows

features) specified within the Docker file. This process is shown in Figure 1 - Docker Image build process.

The nice thing about Docker is that this image-layering principle is reused to optimize the performance and speed of Docker. Once Docker notices that a given layer is already available within the image layer cache on your local machine, it will not download, rebuild or add this layer again. For example, if you have two ASP.NET container images - one for Website 1 and one for Website 2 – Docker will reuse the ASP.NET, IIS and OS layers both at container runtime and in the container image cache. This is shown in Figure 2: How image layering works for containers as well as for container images.F I G U R E 1 : D o c K E R I m a G E b U I L D p R o c E S S








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



F I G U R E 2 : H o w I m a G E L a y E R I n G w o R K S F o R c o n t a I n E R S a S w E L L a S F o R c o n t a I n E R I m a G E S

12

| | | | | | | | | | | | | | | | | | | | | | | EIGHT PRACTICES FOR

Containerized Dellivery on the Microsoft stack

nn

nn

nn

nn

nn

nn

nn

If you implement your container image layers in a smart way, you’ll see an increase in the performance of your containerized workload and the speed of their delivery. Moreover, you’ll see a decrease in the amount of storage your container images require.

The following practices are related to container image layers:• Sequence of layers: Try to order and structure the layering of your container images in such a way that you reuse layers as much as possible. Figure 3 shows how I achieved this for one of my clients by creating different images.

• combining actions in a single instruction: try to combine multiple actions (e.g., by enabling a Windows feature, creating a filesystem directory, etc.) in a single Docker instruction as much as possible. By default, Docker will create a separate container image layer from each individual Docker file instruction. If you don’t need a separate image layer for later use, combine multiple actions in a single instruction line to avoid overhead in the storage of image layers.

For example instead of:

SHELL [“powershell”, “-Command”, “$ErrorActionPreference =

‘Stop’;”]

RUN Add-WindowsFeature NET-Framework-45-Core

RUN Add-WindowsFeature NET-Framework-45-ASPNET

combine both Add-WindowsFeature actions into a single instruction:

SHELL [“powershell”, “-Command”, “$ErrorActionPreference =

‘Stop’;”]

RUN Add-WindowsFeature NET-Framework-45-Core,NET-Framework-

45-ASPNET

Another example of running multiple PowerShell commands in one instruction is:

RUN Invoke-WebRequest “https://aka.ms/InstallAzureCliWindows”

-OutFile az.msi -UseBasicParsing; `

Start-Process msiexec.exe -ArgumentList ‘/i’, ‘C:\

az.msi’, ‘/quiet’, ‘/norestart’ -NoNewWindow -Wait; `

Remove-Item az.msi; `

$env:PATH = $env:AZ_PATH + $env:PATH; `

[Environment]::SetEnvironmentVariable(‘PATH’, $env:PATH,

[EnvironmentVariableTarget]::Machine)

PrACtICe 2: multI-stAge buIldsNew since Docker 17.05 is the concept of multi-stage builds. As described in the previous practice, container images can be rather large because of a large number of image layers. Multi-stage builds help you to reduce the size of your images in an easy way. Instead of an image that contains all in-between image-layers, the multi-stage build implementation makes it possible to copy resulting image content of a given image into another image you define.

Another great benefit of the multi-stage build is that it minimizes your attack surface by removing all in-between layers and installations. And last but not least, multi-stage builds enable easy creation of debug and testing images from your production images by putting the resulting content of your production image into a default debugging and testing image.

Before Docker 17.05 this could also be achieved by making use of the Docker builder-pattern1. In the new multi-stage solution we can just add a second FROM statement to our existing Docker file and instruct that image to copy the content of the earlier stage into the image of this second stage by making use of the --from argument within the COPY instruction. As shown in the following example, the content of the first image definition (called temp) is copied into the second image definition (called final). As a result, the image size of the final image is 315 MB smaller than the original temp image! Looking at the number of image layers, we see a reduction of 15 out of 20 image layers!2

## First Stage – Install WebDeploy, create disk structure,

deploy MVCMusicStore website

FROM microsoft/aspnet AS temp

RUN mkdir c:\install c:\MvcMusicstore

ADD WebDeploy_2_10_amd64_en-US.msi /install/WebDeploy_2_10_

amd64_en-US.msi

WORKDIR /install

RUN msiexec.exe /i c:\install\WebDeploy_2_10_amd64_en-US.

msi /qn

WORKDIR /MvcMusicStore

ADD fixAcls.ps1 /MvcMusicStore/fixAcls.ps1

ADD MvcMusicstore.zip /MvcMusicStore/MvcMusicStore.zip

ADD MvcMusicStore.deploy.cmd /MvcMusicStore/MvcMusicStore.

deploy.cmd

ADD MvcMusicStore.SetParameters.xml /MvcMusicStore/

MvcMusicStore.SetParameters.xml

RUN powershell.exe -executionpolicy bypass .\fixAcls.ps1

RUN MvcMusicStore.deploy.cmd, /Y

## Second Stage - Creates final image

FROM microsoft/aspnet AS final

COPY --from=temp c:/inetpub/wwwroot c:/inetpub/wwwroot

EXPOSE 80

PrACtICe 3: keeP your WIndoWs ContAIners uP-to-dAteIf you work with or start working with Windows containers you may wonder how to implement an update strategy to deal with Windows updates. Because containers are meant to be stateless and immutable, you can’t run Windows Update within your container. So what is the solution?

F I G U R E 3 : R E a L - w o R L D E x a m p L E o F R E U S I n G I m a G E L a y E R S F o R D I F F E R E n t I m a G E S








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



In contrast to Linux container images where people can create a container image from scratch, each Windows container image you create should be based on one of the base images (currently nanoserver3 or windowsservercore4) that were created by Microsoft. Similar to Windows Updates, those base images are updated by Microsoft on a regular basis in order to roll out the latest security and bug fixes. This is one of the reasons why you should always use one of Microsoft’s base images instead of creating your own. As part of the containerized delivery way of- working, you have to put a process in place to ensure that you update your image references to the latest Windows base images on a regular basis.

I recommend my clients to use the docker pull command to automatically pull the latest Microsoft base images each time they have to build an image which is directly dependent on it. This ensures that you always build those container images using the latest OS base images and including all security and bug fixes. For all other container images that depend on those “internal base images”, you have to put a process in place to ensure that they always use one of the most recent internal base images.

Most of the times when I discuss this approach, I get the question: “So we have to download that 10 Gigs of base image each time for each update?” Luckily this is not the case. When you execute the docker inspect command against the base images, you will see that they consist of two layers: one big base layer that will be used for a longer period of time and another smaller update layer that contains small patches and that is updated constantly by Microsoft. So updating to a newer Windows base image version is not painful as it is only pulling the latest update layer from Docker Hub.

PrACtICe 4: grouP mAnAged serVICe ACCountsA lot of existing .NET applications make use of Domain Accounts for authentication, e.g., connecting with SQL Server via Windows Integrated Authentication. The fact that Windows containers cannot be domain-joined may surprise you, but this is by design. Joining a computer to the AD is something that we do for long-term registration. Containers come and go very frequently on a machine and this is not what AD is designed for. Consequently, this could lead to registration issues. Luckily the Windows server team already had a solution in-place to make it possible to make use of Domain Accounts for external authentication from within a container. This solution is called Group Managed Service Accounts (gMSA).

The idea behind this solution is to create a so-called gMSA within Active Directory where you also specify which container hosts have access to this account. This can be a list of container hosts or a security group containing all container hosts that should be able to run a container using this account. The next step is to prepare5 the container hosts and test whether they can access the gMSA. After that, you have to install and make use of the CredentialSpec

PowerShell module to store the required gMSA details in a correct (JSON credential spec) format on your container hosts.

Once the required gMSA details have been stored, you can make use of the stored gMSA credentials spec to authenticate external services under the given gMSA from within your container. The only thing you have to ensure within your container is that your services and other processes that need to access the external resource run as ‘Local System’ or ‘Network Service’. When you start your container on the container host you can execute a docker run command with the --security-opt “credentialspec=file://WebApplication1.json” argument in order to pass the appropriate credential spec on to your container. At the same time, the applications within your container can access the external resource. For an extensive walk-through, please refer to http://bit.ly/2tWGloy.

PrACtICe 5: seCure ContAInerIzed delIVerySecuring your container infrastructure and deployments is an important aspect of Containerized Delivery. There are a lot of aspects6 to keep in mind here, so I will highlight the most important ones.• Harden your images, containers, daemons and hostsWhen you set up your containerized infrastructure, it is important that you harden your infrastructure elements against threats. To help you with this, the Center for Internet Security has published a Docker benchmark that includes configuration and hardening guidelines for containers, images, and hosts. Have a look at this benchmark at https://www.cisecurity.org/benchmark/docker/.

Based on this benchmark, there’s a Linux container available that checks for dozens of common best-practices around deploying Docker containers in production in a scripted way. Unfortunately, this implementation is not available for Windows hosts right now, but if you make use of Linux container hosts for your ASP.NET core applications, you should definitely check this implementation at https://github.com/docker/docker-bench-security.

One important aspect of hardening your container hosts is protecting your Docker daemon with TLS. A great, fast and simple way to achieve this is to use Stefan Scherer’s dockertls-windows container7. This container generates all TLS certificates you’ll need to access the secured container daemon. Save the .pem files in a central, secure location so that you can use the content of those files once you want to access the secured Docker daemon. If you make use of VSTS for CI/CD, you can store the contents of the various .pem files directly in the Docker host Service endpoint.

• Know the origin and content of your images As mentioned in practice 3, there are two Microsoft base images that all Windows container images should derive from. However, there are









FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



F I G U R E 4

F I G U R E 5

F I G U R E 6 : G m S a c o n t a I n E R a U t H E n t I c a t I o n F L o w

CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



14



nn

nn

nn

nn

nn

nn

nn

14

| | | | | | | | | | | | | | | | | | | | | | |EIGHT PRACTICES FOR

Containerized Delivery on the Microsoft stack

nn

nn

nn

nn

nn

nn

nn

also a lot of other public container images available on DockerHub, such as microsoft/iis, microsoft/powershell and even images from other publishers. Using those out-of-the-box images accelerates the development of your systems. However, making use of public image definitions can expose your production landscape to high risk. For example, how can you make sure that those images do not contain any vulnerabilities? How do you ensure that the owner of the image will maintain the image definition over time in case of vulnerabilities and exploits that are discovered? It is important to know the origin and content of the images you consume. Luckily there are many tools available to help you fill this gap. For example, you can make use of Docker Notary to check the authenticity of images, or Docker Security Scan to scan for any vulnerabilities within your image. You can also use other solutions such as Aqua8 and Twistlock9. Whatever tool you use, make sure that you put a process in place that forces you to only use scanned public images and trusted origins.

For existing internal images, it is important that you perform regular checks and actively maintain those images with regard to vulnerabilities and exploits. For new internal images it is important that you reduce the attack surface as much as possible. Many of the out-of-the-box images from DockerHub, such as the microsoft/iis and microsoft/aspnet enable too many features for your workload. At one of my clients, this was the reason that we decided to create our own internal IIS base image with only those Windows features and services enabled that we really needed. For example, the default IIS image enables all Web-Server sub-features. The image we created enables only some of the sub-features, e.g., Web-Static-Content, Web-Http-Logging, Web-Stat-Compression, and Web-Dyn-Compression. By creating our own internal IIS image we made our workload more secure and achieved better performance. To find out which Windows features you really need, take a look at the Power Shell Get-WindowsOptionalFeature –Online and Get-WindowsFeature commandlets.

PrACtICe 6: deAlIng WIth seCretsBefore the container era, we used to put our secrets (i.e.,credentials, certificates, connection strings) at a given location in the file system, alongside our application files. Within a containerized world, there is a problem with this approach. Because container images contain both the application as registry, environment variables and other file system content, this approach would mean that each team member can see the secrets by spinning up a new container based on this image. Dealing with secrets of containerized applications therefore means that you need to specify your secrets on container initialization and store them outside your container images. But how can you achieve this?

Before we look at the solution for dealing with secrets, you have to know exactly what you need to declare as secrets. Many values in configuration files are not secrets, e.g., endpoints, whereas passwords and SSL certificates are definitely secrets. It is important to be aware of this separation between secrets and configuration settings because in an ideal world you will manage both in a different way.

Looking at configuration settings, there are several ways to manage them. The option I like most is to make use of a configuration container in which all configuration settings (e.g., endpoints) are

stored. At the time of container initialization, you can make use of this container to get the right endpoints for your application, for instance a service bus topic endpoint or an external SMS endpoint. The nice thing about a configuration container is that you don’t have to change the content of all other containers to deal with configurations over multiple environments like Dev, Test and Production. By making use of Docker Compose you can define this configuration store as a separate service and use its Docker DNS name to get the latest configuration settings from that store.

Until the beginning of this year, the most frequently used solution for dealing with secrets was either to make use of volume mappings or to make use of environment variables that contain the actual secrets. However, neither of these options was very secure. In the case of environment variables, your secrets are accessible by any process in the container, preserved in intermediate layers of an image, visible in docker inspect, and shared with any container linked to the container. In the case of volume mappings, the disadvantage is that you are making your containers dependent on the content of a data volume and this means that this container becomes unnecessarily stateful instead of stateless.

Luckily, since the beginning of this year, the most applicable option is to make use of the secrets management solution of the different cluster implementations, e.g., Kubernetes Secrets or Docker Secrets (Docker 17.05 for Windows containers). The nice thing about secret management at the cluster level is that secrets are automatically distributed across the container hosts. Another benefit is that the same secret name can be used across multiple clusters. If you have a separate Development, Test and Acceptance cluster, you can reuse the secret name, and your containers only need to know the name of the secret in order to function in all three environments. Creating those secrets in your container cluster environment can be orchestrated by the tools you are using for your delivery pipeline, e.g.,VSTS.

PrACtICe 7: exPlICIt dePendenCy mAnAgementA lot of Docker and Compose files I have seen contain a reference to images, without specifying the version of the image, or with the default “latest” tag. Not explicitly specifying the version of an image can result in unexpected behavior and this is very risky.

The reason for this is that every Docker host has its own local Docker image cache. Once an image-tag combination is found in your local cache, Docker will consume the cached blob on docker run instead

F I G U R E 7

F I G U R E 8








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



You can leverage from the above characteristics by extending your existing set of individual delivery pipelines with a combined pipeline that can deploy your entire application stack to production. You’ll still have a separate commit (build) stage for each individual application, but the other delivery pipeline stages (e.g., automated acceptance test and user acceptance test stages) are combined stages within this pipeline. At the end of the pipeline you deploy your entire application stack to production. As a result, you can choose wether you want to deploy your entire application stack at once, or whether you just want to deploy the newest version of a separate application.

of downloading the blob from the container registry. Making use of the “latest” tag could result in the fact that you get an older version of an image than the image you actually want. As shown in Figure 9 a docker run cornellk/test command on Host01 will run version 10.0.01 instead of the download of image version 10.0.02 from the registry.

You can test this scenario yourself by comparing the result of a docker run and docker pull command on an outdated image in the local cache. As you see in Figure 10, docker pull downloads newer image layers from the registry where docker run just starts a new container based on the outdated cached image. Using the “latest” tag makes your build-and-release process non-repeatable. To avoid this, all you need to do is explicitly reference which version of an image you want.

PrACtICe 8: enVIronment-As-Code PIPelIne And IndIVIduAl PIPelInesContainerized Delivery means that you treat your application stack as cattle instead of pets10. This means that you deal with the immutability and stateless characteristics of containers. The nice thing about those characteristics is that they make your containerized application stack reproducible and scalable. Instead of upgrading, you replace your application stack.


CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



F I G U R E 9 : t H E L a t E S t ” t a G I S S U E

F I G U R E 1 0 : D o c K E R R U n V S D o c K E R p U L L I n c a S E o F I m p L I c I t V E R S I o n I n G

F I G U R E 1 1 : D E L I V E R y p I p E L I n E S I n a c o n t a I n E R I Z E D w o R L D

Cornell knulst

Cornell is a Cloud Software Architect and ALM consultant at Xpirit. He enables companies to deliver software faster, cheaper and better by sup-plying guidance on adopting DevOps practices and migrating to modern, cloud-native architectures. He is passionate about new technologies and is continuously diving into the latest technologies. He actively maintains a personal blog, speaks regularly at both national and international industry events and regularly publishes articles in national trade magazines.

CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL

1 https://blog.alexellis.io/3-steps-to-msbuild-with-docker/ 2 http://fluentbytes.com/optimizing-your-windows-docker-images-with-multi-staged-docker-builds/

3 https://hub.docker.com/r/microsoft/nanoserver/

4 https://hub.docker.com/r/microsoft/windowsservercore/

5 https://github.com/Microsoft/Virtualization-Documentation/tree/live/windows-server-container-tools/ServiceAccounts

6 https://www.twistlock.com/resources/secops-guide-container-security/

7 https://stefanscherer.github.io/protecting-a-windows-2016-docker-engine-with-tls/

8 https://www.aquasec.com/ 9 https://www.twistlock.com 10 https://www.theregister.co.uk/2013/03/18/servers_pets_or_cattle_cern/

Cloning FDDataSet Cursors

Cary Jensen

FireDAC about these issues, he explained that CloneCursor was intended for use only from FDMemTable.

The following is the declaration of CloneCursor:

procedure CloneCursor(ASource: TFDDataSet;

AReset: Boolean = False;

AKeepSettings: Boolean = False);

virtual;

When you invoke CloneCursor, the first argument that you pass is a reference to an active FDDataSet whose cursor you want to clone. The AReset and AKeepSettings are used to either keep or discard the original FDDataSet’s view. If you pass a value of False in both of these parameters, the cloned cursor will adopt the values of the DetailFields, Filter, Filtered, FilterOptions, FilterChanges, IndexName, IndexFieldNames, MasterSource and MasterFields properties, as well as the OnFilterRecord event handler, of the source dataset. If AReset is True, these properties will assume their default values. If AReset is False and AKeepSettings is True, the clone will assume these properties, but they may or may not be entirely valid. For example, the clone may be set to an index that exists on the source but not on the destination.

Once you have cloned the cursor of an existing FDDataSet to an FDMemTable, there are two specific uses. The first, and most general, is to use the clone as an additional, readonly view of the data. This view can have a different current record, sort order (index), range, and filter.

The second use is as an editable cursor into the cloned FDDataSet’s data store. This use, however, has some important limitations. First, if the dataset you are cloning is not an FDMemTable, the cloned FDDataSet must be in a cached updates mode. Second, you can only apply those updates by calling the ApplyUpdates method of the originally cloned FDDataSet. Failure to call ApplyUpdates on the original dataset will result in the loss of updates applied either by the original FDDataSet or by any of its clones.

I demonstrate the use of cloned cursors in the following three examples. The first example demonstrates a read-only clone that provides a master-detail view on a single table. The second example demonstrates an editable clone of an FDDataSet in cached updates mode. The final example demonstrates editing a FDMemTable using a cloned cursor.

oVerVIeW of Cloned CursorsWhat do you do when you need two different views of the same data at the same time? One alternative is to load two copies of the data into memory. This approach, however, results in an unnecessary increase in network traffic (or disk access) and places redundant data in memory.

In some cases, a better option is to clone the cursor of an already populated FDDataSet. When you clone a cursor, you create a second, independent pointer to an existing FDDataSet’s memory store, including Delta (the change cache, if cached updates are being employed). Importantly, the cloned FDDataSet has an independent view, including, but not limited to, current record, filter, index, and range.

It is difficult to appreciate the power of cloned cursors without actually using them, but an example can help. Imagine that you have loaded 25,000 records into an FDMemTable, and you want to compare two separate records in that FDMemTable programmatically.

One approach is to locate the first record and save some of its data into local variables. You can then locate the second record and compare the saved data to that in the second record.

Yet another approach is to load a second copy of the data in memory. You can then locate the first record in one FDMemTable, the second record in the other FDMemTable, and then directly compare the two records.

A third approach, and one that has advantages over the first two, is to utilize the one copy of data in memory, and clone a second cursor onto this memory store. The cloned FDMemTable cursor appears as if it were a second copy of the data in memory, in that you now have two cursors (the original and the clone), and each can point to a different record and utilize a different index. Importantly, only one copy of the data is stored in memory, and the cloned cursor provides a second, independent pointer into it. You can then point the original cursor to one record, the cloned cursor to the other, and directly compare the two records.

The CloneCursor method, is introduced in the FDDataSet class, which means that it can be called by any FDDataSet, not just FDMemTables. For the most part, however, you will not call CloneCursor with any FDDataSet other than FDMemTables. Trying to do so often leads to access violations and other issues. When I asked the author of

Some time ago I wrote about cloning cursors with ClientDataSets, which at the time was the only component that shipped with Delphi that supported this technique. That changed with the introduction of FireDAC in Delphi XE3. All FireDAC datasets (FDDataSets), including FDQueries, also support cloned cursors, though this support is somewhat limited compared to ClientDataSets. In this article, which is based on material that I included in my most recent book, Delphi in Depth: FireDAC, I discuss cloning FireDAC cursors, with particular focus on FDMemTables.









FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



18



nn

nn

nn

nn

nn

nn

nn

18

| | | | | | | | | | | | | | | | | | | | | | |Cloning

FDDataSet Cursors

//Clone the detail cursor.

OrdersByPartNoMemTable.CloneCursor(PartsQuery, False);

//Setup the master-detail relationship

OrdersByPartNoMemTable.MasterSource := PartsSource;

OrdersByPartNoMemTable.MasterFields := ‘PartNo’;

OrdersByPartNoMemTable.IndexFieldNames := ‘PartNo’;

OrdersByPartNoMemTable.IndexesActive := True;

//The PartsSource.OnDataChange event needs Filtered to be

True

OrdersByPartNoMemTable.Filtered := True;

end;

Figure 2 shows the running form. The cloned cursor is shown in the lower DBGrid. Since this cloned cursor has been configured to act as the detail table in a master-detail relationship, the clone only displays those orders where the part number is the same as that associated with the current record in the upper DBGrid.

If you inspect Figure 2, you will see that the current record in the master table is associated with PartNo 12310. Furthermore, the detail table is displaying all records in the dataset in which the PartNo is 12310.

The main form also includes a checkbox, whose caption is Include Current OrderNo. In Figure 2, you can see that the current record in the upper DBGrid is associated with order number 1004, and the detail records include that order number (as well as order number 1074). If you uncheck the checkbox, the detail dataset will include only the other orders that contain the selected part number in the master table, which in this case, would cause order number 1004 to be omitted from the detail view.

This effect is controlled by the OnDataChange event handler of DataSource1, whose DataSet property points to PartsQuery. Each time you navigate to another record in the master table, this event handler triggers to update the Filter property of the detail table, as shown in the following code:

nn

nn

nn

nn

nn

nn

nn

mAster WIth detAIl CloneThis example, one that I really like, is based on a cloned cursor example that I originally included in my two previous ClientDataSet books, Delphi in Depth: ClientDataSets (first and second editions). In this example, I use a single query result set to display a self-referencing master-detail relationship. The main form of this project is shown in Figure 1.

FDQuery1 holds a simple query of the Items table from the dbdemos.gdb database that is found in Delphi’s sample database. Here is the SQL associated with this query:

SELECT * FROM Items

The OnCreate event handler of the form modifies the connection found on the SharedDMVcl data module to point to the dbdemos.db database (note that I’m not using the employee.gdb database that is used in most of the code samples in this book). This is the connection that the FDQuery, named PartsQuery, is connected to. Next, the FDMemTable, named OrdersByPartNoMemTable, is used to clone the query, after which OrdersByPartNoMemTable is configured as a detail table of PartsQuery. The final step in this OnCreate event handler is to set the Filtered property of the FDMemTable to True, which will initially have no effect since the Filter property is blank. Here is that event handler:

procedure TForm1.FormCreate(Sender: TObject);

var

DBIndex: Integer;

begin

//Set the connection to use the dbdemo.gdb database

DataPaths.ValidatePath( IBDemoPath );

// Switch database for

DBIndex :=

SharedDMVcl.FDConnection.Params.IndexOfName(‘Database’);

if DBIndex > -1 then

SharedDMVcl.FDConnection.Params[DBIndex] :=

‘Database=localhost:’ + DataPaths.IBDemoPath;

// Get the Items table records

PartsQuery.Open;

F I G U R E 1 : t H E F D m a S t E R D E t a I L c L o n E p R o J E c t m a I n F o R m

F I G U R E 2 : a S I n G L E I n - m E m o R y D a t a S t o R E I S U S E D t o D I S p L a y b o t H t H E m a S t E R R E c o R D S a S w E L L a S t H E D E t a I L R E c o R D S








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



procedure TForm1.DataSource1DataChange(Sender: TObject;

Field: TField);

begin

if not IncludeCurrentOrderCbx.Checked then

OrdersByPartNoMemTable.Filter := ‘OrderNo <> ‘ +

QuotedStr(PartsQuery.FieldByName(‘OrderNo’).AsString)

else

OrdersByPartNoMemTable.Filter := ‘’;

end;

Clicking the checkbox also results in a change to the Filter property of the cloned dataset. However, in this case, it does so by directly calling the OnDataChange event handler of DataSource1, as shown in the following code:

procedure TForm1.IncludeCurrentOrderCbxClick(Sender:

TObject);

begin

DataSource1DataChange(Self, PartsQuery.Fields[0]);

end;

edItIng WIth Cloned Cursors And CAChed uPdAtesCached updates is a special mode supported by FDDataSets. When in the cached updates state, edits made to an FDDataSet are not immediately written to the underlying database. Instead, they are held in memory until that memory is cleared, or until there is an explicit call to ApplyUpdates, at which point the FDDataSet will attempt to write all of the changes in cache to the underlying database.

When using cached updates, it is possible to create two or more cloned cursors that can write and manage changes to the cached updates state of the FDDataSet that is cloned. This is a powerful capability, and one that suits FDMemTables very well.

In this section, I am going to demonstrate how to provide two or more editable cursors into a single FDQuery using cloned cursors and cached updates. Cached updates provide you with a rich set of options for editing data, and I’ve discussed cached updates with FireDAC in a previous article. For that reason, I am going to limit the

discussion of cached updates in this chapter to entering the cached updates state and applying the cached updates to the underlying database, with my primary focus on the cloning of cursors.

Editing with cloned cursors and cached updates is demonstrated in the project named FDCloningCachedCursors. The main form of this project is shown in Delphi’s designer in Figure 3.

This project contains one FDQuery and two FDMemTables. The FDQuery contains a simple SELECT * FROM Employee SQL statement, and is opened by clicking on the button whose default caption is Open FDQuery1. This button, when clicked, will determine whether the query is active or not. If currently closed, the query will be opened and will be placed in the cached updates state, so long as the checkbox labeled Cached Updates is checked. If the query is already active, it is closed. Here is the code associated with the OnClick event of this button:

procedure TMainform.btnOpenQuery1Click(Sender: TObject);

begin

if not FDQuery1.Active then

begin

FDQuery1.Open();

FDQuery1.CachedUpdates := cbxCachedUpdates.Checked;

btnApplyUpdates.Enabled := cbxCachedUpdates.Checked;

btnOpenQuery1.Caption := ‘Close FDQuery1’;

end

else

begin

FDQuery1.Close;

btnApplyUpdates.Enabled := FDMemTable1.Active;

btnOpenQuery1.Caption := ‘Open FDQuery1’;

end;

end;

Once FDQuery1 has been opened, FDMemTable1 or FDMemTable2 or both can clone the FDQuery1 cursor. At any time, the data originally retrieved through the FDQuery1 SQL statement can be viewed, and even edited, by any of the active datasets.

This can be seen in Figure 4. Here FDQuery1 has been opened, and both FDMemTable1 and FDMemTable2 have cloned the FDQuery1 cursor. In Figure 4, the lower DBGrid which is associated with FDMemTable2, has been edited, changing the value in the first name field of the first record from Robert to Bob. Once this change has been posted to FDMemTable2, the update can immediately be seen in all three grids, and the calculated field in FDQuery1, which displays the value of the UpdateStatus property for each record in memory, now shows that this record has been modified.

If you take a close look at Figure 4, you might notice that the Cached Updates checkbox is not checked. As a result, although FDQuery1 displays the updated data, and the status of the record appears as usModified, the query was not aware that a change has been posted, and so it did not write this change to the underlying database upon posting. Unless FDQuery1 is in cached updates mode when a clone edits the data, the changes made by the clone will not be respected.









FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL

F I G U R E 3 : t H E m a I n F o R m o F t H E F D c L o n I n G -c a c H E D c U R S o R S p R o J E c t








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



20



nn

nn

nn

nn

nn

nn

nn

| | | | | | | | | | | | | | | | | | | | | | |Cloning

FDDataSet Cursors

nn

nn

nn

nn

nn

nn

nn

entering the cached updates mode. This is demonstrated in the project named DBGridDragDropFDMemTable, whose main form is shown in Figure 5.

This project makes use of a cloned FDMemTable to enable a drag-drop operation within, and to, a DBGrid, and relies on the ability of a cloned FDMemTable to edit the FDMemTable that it cloned in order to update the order of records in the DBGrid.This project is somewhat complicated, and a complete description of it is beyond the scope of this article. However, I have written about this project in my blog, Let’s Get Technical, twice. For more information on how and why this technique works, you should visit http://caryjensen.blogspot.com/2013/08/dragging-and-dropping-in-dbgrids.html, as well as its preceding post at http://caryjensen.blogspot.com/2012/08/dragging-and-dropping-into-dbgrids.html.

The three critical methods that employ cloned cursors are shown here:

procedure TForm1.MoveRecord(MemTable: TFDMemTable; OldPos, NewPos: Integer);var clone: TFDMemTable; SequenceFld: TField;begin clone := TFDMemTable.Create(nil); try clone.CloneCursor(MemTable, True); SequenceFld := clone.FieldByName(‘Sequence’); clone.RecNo := OldPos; clone.Edit; //Move the record being moved to the end of the sequence SequenceFld.AsInteger := MemTable.RecordCount + 1; clone.Post; if OldPos < NewPos then begin //Shift the records after the original old position up one position

clone.RecNo := OldPos; while (clone.RecNo < NewPos) do begin clone.Edit; SequenceFld.AsInteger := SequenceFld.AsInteger - 1; clone.Post;

This is a bit complicated, so let me try to be as clear as possible. If FDQuery1 is open in cached updates mode, and then its cursor is cloned, changes made by the clone will appear in the change log of FDQuery1, and will be applied when ApplyUpdates is called on FDQuery1. If FDQuery1 is not in cached updates mode, edits made by a cloned cursor will appear as though they are in FDQuery1, but will not be written to the database. Similarly, if FDQuery1 is placed into cached updates mode after a clone has edited the data, the clone’s changes made prior to FDQuery1’s entry into cached updates mode will not be in the change cache, and therefore will not be seen as an edited record that needs to be applied.The only way that a clone can update the shared data store is to post an edit to the data while FDQuery1 is in cached updates mode, after which a call to FDQuery1’s ApplyUpdates method will attempt to write that change to the underlying database. Only under that condition will FDQuery1’s records edited by a clone be marked with the update status information necessary for ApplyUpdates to apply the clone’s edits.

Having said that, any changes made directly to FDQuery1, either programmatically or though data binding, when it is not in cached updates mode are immediately written to the underlying database once that change is posted to FDQuery1, whether or not any clones of FDQuery1 exist. And, any changes made directly to FDQuery1 while it is in cached updates mode will reside in the change cache, and a subsequent call to ApplyUpdates will attempt to write that data to the underlying database. Furthermore, only the original FDDataSet can write the cached updates to the underlying database. Calling ApplyUpdates on a cloned cursor will have no effect.Being able to edit a common data store using one or more cloned cursors and cached updates is a powerful capability, one that can provide the basis for sophisticated features within your applications. However, this is a complicated topic. Playing around with the FDCloningCachedCursors project can be a good way to better understand how this mechanism works.

edItIng fdmemtAbles usIng A Cloned CursorWhen using an FDMemTable to clone the cursor of another FDMemTable, it is possible to use the clone to edit data without

F I G U R E 4 : F D m E m t a b L E 2 , a c L o n E D c U R S o R o F F D Q U E R y 1 , H a S b E E n E D I t E D . t H E p o S t E D c H a n G E S a R E n o w V I S I b L E I n b o t H F D m E m t a b L E 1 ( a n o t H E R c L o n E ) a n D F D Q U E R y 1

F I G U R E 5 . t H E m a I n F o R m o F t H E D b G R I D D R a G -D R o p F D m E m t a b L E p R o J E c t








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



20

end;end;

procedure TForm1.ResequenceTable(MemTable: TFDMemTable; FromRow: Integer);var clone: TFDMemTable; SequenceFld: TField;begin clone := TFDMemTable.Create(nil); try clone.CloneCursor(MemTable, True); SequenceFld := clone.FieldByName(‘Sequence’); begin //Shift all records down to make room in the sequence //for the record being inserted clone.Last; while (SequenceFld.AsInteger >= FromRow) and not clone.bof do begin clone.Edit; SequenceFld.AsInteger := SequenceFld.AsInteger + 1; clone.Post; clone.Prior; end; end finally clone.Free; end;end;

summAryFireDAC FDDataSets, like Delphi’s ClientDataSet, support cloned cursors. With FireDAC, you use an FDMemTable to clone the cursor of any FDDataSet. When cloning FDDataSets other than FDMemTables, the clone can be used in a readonly mode, or can be used to edit, so long as the FDDataSet that is cloned is in a cached updates mode. When the FDDataSet that is cloned is an FDMemTable, the clone can be used in read or write modes.This article was adapted from Chapter 13: More FDMemTables: Cloned Cursors and Nested DataSets from my latest book - Delphi in Depth: FireDAC. This book is currently available in both print and ebook editions. For more information, please visit: http://www.JensenDataSystems.com/FireDACBookYou can download the source code for the projects discussed in this article at http://www.JensenDataSystems.com/CloningFDData Sets.zip.

clone.Next; end; end else begin //Shift the record before the original position down one position clone.IndexFieldNames := ‘Sequence’; clone.RecNo := OldPos - 1; while (clone.RecNo >= NewPos) and (not clone.bof) do begin clone.Edit; SequenceFld.AsInteger := SequenceFld.AsInteger + 1; clone.Post; clone.Prior; end; end; //Move the record being moved to its new position clone.RecNo := MemTable.RecordCount; clone.Edit; SequenceFld.AsInteger := NewPos; clone.Post; finally clone.Free; end; MemTable.RecNo := NewPos;end;

procedure TForm1.RemoveFromSequence(MemTable: TFDMemTable; Position: Integer);var clone: TFDMemTable; SeqFld: TField;begin clone := TFDMemTable.Create(nil); try clone.CloneCursor(MemTable, True); SeqFld := clone.FieldByName(‘Sequence’); clone.RecNo := Position; if (Position = 1) and (clone.RecordCount = 1) then begin //There is just one record. Delete it, //but do not try to set a new record position clone.Delete; end else begin if clone.RecNo = clone.RecordCount then begin clone.Delete; MemTable.RecNo := MemTable.RecordCount; end else begin clone.Delete; while not clone.eof do begin clone.Edit; SeqFld.AsInteger := SeqFld.AsInteger - 1; clone.Post; clone.Next; end; MemTable.RecNo := Position; end; end; finally clone.Free;

CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL









FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



CAry Jensen

Cary is Chief Technology Officer at Jensen Data Systems, Inc., a company that has been providing training, consulting, and software development services since 1988. He is an award-winning author and trainer. His latest book is Delphi in Depth: FireDAC. He is an active developer, providing clients with assistance in data modeling, software architecture, software development, team development, mentoring, training, and software migration. Cary has a Ph.D. in Engineering Psychology, from Rice University, specializing in human-computer interaction

CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL









FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



EEN SIMPEL SySTEEM VOOR BEELDHERKENNING MET

Microsoft Computer Vision API en Solr

Roel Hans Bethlehem

Wat heb je hiervoor nodig:• EenWindowsmachineomopteontwikkelen• Optioneel:eenmachinemetLinuxeropomSolrtedraaien, mag ook op dezelfde Windows machine. In het artikel gebruik ik een aparte Linux VM op Hyper-V.• MicrosoftVisualStudio2017(ikgebruikeen.NETCoreconsole app)• ToegangtotdeCognitiveServices(ComputerVisionAPIis onderdeel daarvan)

Het is ook mogelijk om in plaats van .NET en Visual Studio bijvoorbeeld Python te gebruiken. Het meeste werk wordt verzet via HttpClient in .NET maar curl in de Linux wereld werkt ook prima.

De user story is als volgt: Als gebruiker wil ik een foto maken met bijvoorbeeld de camera van mijn smartphone en deze filmposter vinden.

Als voorbeeld heb ik hier een filmposter op het internet gezocht:https://nei1b.deviantart.com/art/Star-Wars-VI-Return-of-the-Jedi-Movie-Poster-351194269

Nu is iets herkennen makkelijk voor een mens. We zien dat de titel Star Wars en een Ewok dus moet het wel Return of the Jedi zijn en de ondertitel geeft het natuurlijk ook wel weg.

Stel je voor je bent een computer hoe weet je dan om welke poster het gaat? Je hebt geen referentiekader van schattige Ewoks, Han Solo, Darth Vader en X-Fighters dus je hebt wat hulp nodig om allerlei dingen zoals woorden, kleuren, plekken en personen te herkennen. Microsoft Cognitive Services en specifiek de Computer Vision API om informatie over woorden, kleuren, plekken en personen aan het plaatje te ontfutselen.

Om aan de hand van de woorden, plekken, kleuren en personen het plaatje te kunnen matchen kunnen we Solr gebruiken. Apache Solr is een zoekmachine waarmee we een taxonomie kunnen ontwikkelen om vervolgens het meest relevante plaatje te vinden.

Eerst gaan we eens aan de slag met de Computer Vision API om de filmposter die ik uitgekozen had te analyseren.

De onderstaande code (in bijvoorbeeld een console app) verstuurt een request naar de OCR (Optical Character Recognition) service die onderdeel is van de Computer Vision API en retourneert een scaninfo object. Een scaninfo object bestaat uit regions die weer onderverdeeld zijn in lines die bestaan uit words. Ik vind het makkelijk om met een .NET object te werken en map daarom de resulterende json response op mijn eigen scaninfo object. De subscriptionkey kun je verkrijgen door een trial account (zie onder in de sectie links).

private const string uriOcrBase = “https://westcentralus.api.cognitive.microsoft.com/vision/v1.0/ocr”;

static void Main(string[] args){ Console.WriteLine(“Optical Character Recognition:”); Console.Write(“Enter the path to an image with text you wish to read: “); string imageFilePath = Console.ReadLine(); byte[] bytes = GetImageAsByteArray(imageFilePath); var r = MakeOCRRequest(bytes, uriOcrBase).Result;}

public static async Task<ScanInfo> MakeOCRRequest( byte[] byteData, string uriOcrBase){ HttpClient client = new HttpClient(); // Request headers.

client.DefaultRequestHeaders.Add (“Ocp-Apim-Subscription-Key” , subscriptionKey);









FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



In dit artikel leg ik uit hoe je een eenvoudige zoekmachine voor plaatjes kunt maken met behulp van de Microsoft Computer Vision API en Apache Solr als zoekmachine. Je leert dus zowel iets over de nieuwe Cognotive Services waarvan de Computer Vision API onderdeel is alsook over de zoekmachine Solr.








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL

F I G U R E 1








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



24 magazine voor software development 24

Caesar praemuniet Augustus.Zothecas insectat gulosus oratori. Tremulus saburre celeriter amputat ossifragi, ut utilitas fiducias corrumperet saetosus saburre. Suis prae-muniet Octavius. Apparatus bellis pessimus frugaliter fermentet vere-cundus oratori, utcunque Medusa agnascor aegre utilitas saburre. Cathedras iocari pretosius suis, ut syrtes incredibiliter comiter praemu-niet oratori. Concubine vocificat syrtes, etiam chirographi suffragarit cathedras. Catelli deciperet fragilis fiducias. Ossifragi verecunde in-sectat apparatus bellis. Vix quinquennalis catelli deciperet suis, quod quadrupei insectat incredibiliter adlaudabilis zothecas.Catelli imputat agricolae, semper matrimonii circumgrediet adfabilis rures, iam ossifragi suffragarit utilitas saburre. Adlaudabilis fiducias deciperet agricolae. Bellus zothecas conubium santet adlaudabilis apparatus bellis, etiam Caesar frugaliter fermentet oratori. Augustus spinosus corrumperet chirographi.Gulosus saburre aegre infeliciter insectat ossifragi. Satis perspicax syrtes fermentet parsimonia agricolae, quamquam adlaudabilis sa-burre insectat plane bellus agricolae. Incredibiliter utilitas umbraculi fortiter imputat pessimus quinquennalis suis, utcunque adfabilis con-cubine insectat zothecas, ut Aquae Sulis verecunde imputat perspi-cax umbraculi. Pompeii infeliciter vocificat concubine, semper saburre deciperet suis. Augustus vocificat Aquae Sulis, quod oratori optimus neglegenter suffragarit aegre gulosus ossifragi. Pretosius quadrupei

amputat incredibiliter lascivius rures, etiam optimus pretosius chi-rographi deciperet concubine. Fragilis catelli lucide circumgrediet ca-thedras, iam perspicax concubine amputat lascivius rures. Fiducias iocari concubine. Syrtes divinus praemuniet satis quinquennalis rures, et verecundus apparatus bellis conubium santet quadrupei, iam rures fermentet oratori.Optimus adlaudabilis agricolae vocificat syrtes, quamquam adfabilis oratori imputat pretosius saburre, et Medusa libere agnascor agri-colae, utcunque oratori corrumperet aegre quinquennalis quadrupei. Augustus conubium santet suis, ut pretosius zothecas circumgrediet umbraculi. Pompeii adquireret suis, quod concubine vocificat cathe-dras. Catelli corrumperet tremulus suis, ut zothecas conubium santet bellus matrimonii. Verecundus umbraculi comiter miscere rures, sem-per chirographi corrumperet MedusaOctavius praemuniet agricolae. Matrimonii iocari concubine.

Parsimonia saburre adquireret optimus gulosus quadrupei. Fragilis catelli conubium santet gulosus cathedras, semper vix lascivius rures circumgrediet quadrupei, quamquam adlaudabilis syrtes infeliciter agnascor chirographi, iam zothecas pessimus comiter adquireret ap-paratus bellis.Incredibiliter adfabilis catelli celeriter conubium santet Pompeii. Zothe-cas fermentet concubine, quamquam utilitas syrtes plane libere im-putat cathedras. Rures suffragarit verecundus chirographi, utcunque Octavius senesceret syrtes. Incredibiliter gulosus matrimonii adquir-eret rures. Satis adfabilis catelli incredibiliter comiter insectat oratori.Verecundus concubine fermentet catelli, ut utilitas rures vix lucide cor-rumperet saetosus quadrupei, iam chirographi plane fortiter circum-grediet pessimus pretosius oratori.

Saburre conubium santet verecundus concubine, semper chirographi amputat cathedras, quod quadrupei praemuniet chirographi.\ Optimus lascivius apparatus bellis iocari quinquennalis ag-ricolae, et adfabilis oratori deciperet verecundus syrtes

CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL

Denfeliciter insectat ossifragi. Satis perspicax syrtes fermentet parsimonia agricolae, qua-mquam adlaudabilis saburre insectat plane bellus agricolae.

SDN

| | | | | | | | | | | | | | | | | | | | | | |

nn

nn

nn

nn

nn

nn

nn

// Request parameters.

string requestParameters =

“language=unk&detectOrientation=true”;

// Assemble the URI for the REST API Call.

string uri = uriOcrBase + “?” + requestParameters;

HttpResponseMessage response;

using (ByteArrayContent content =

new ByteArrayContent(byteData))

{

// This example uses content type “application/octet-stream”.

// The other content types you can use are “application/json”

// and “multipart/form-data”.

content.Headers.ContentType =

new MediaTypeHeaderValue(“application/octet-stream”);

// Execute the REST API call.

response = await client.PostAsync(uri, content);

// Get the JSON response.

string contentString =

await response.Content.ReadAsStringAsync();

ScanInfo scaninfo = new ScanInfo();

scaninfo = Newtonsoft.Json.JsonConvert

.DeserializeObject<ScanInfo>(contentString);

return scaninfo;

}

}

De output van de OCR service is dit:

{

“language”: “en”,

“textAngle”: 0.0,

“orientation”: “Up”,

“regions”: [

{

“boundingBox”: “233,739,275,288”,

“lines”: [

{

“boundingBox”: “266,739,234,50”,

“words”: [

{

“boundingBox”: “266,739,234,50”,

“text”: “WAu”

}

]

},

{

“boundingBox”: “258,800,250,14”,

“words”: [

{

“boundingBox”: “258,800,85,14”,

“text”: “RETUR.N”

},

{

“boundingBox”: “361,800,39,14”,

“text”: “OF.”

},

{

“boundingBox”: “405,801,39,13”,

“text”: “THE”

},

{

“boundingBox”: “460,800,48,14”,

“text”: “JEDI”

}

]

... (ingekort)

{

“boundingBox”: “316,1021,134,6”,

“words”: [

{

“boundingBox”: “316,1021,5,6”,

“text”: “A”

},

{

“boundingBox”: “402,1021,15,6”,

“text”: “rox”

},

{

“boundingBox”: “419,1021,31,6”,

“text”: “RELEASE”

}

]

}

]

}

]

}

Niet slecht, het lijkt op Return of the Jedi. Ook zien we aan de boundingBox waar het woord ongeveer staat en aan de lines welke woorden bij elkaar horen. Zo ingewikkeld gaan we het hier nu niet maken want we gaan nu enkel aan de hand van de woorden proberen de poster te matchen met posters in de zoekmachine Solr.

Op mijn machine heb ik Hyper-V met een Ubuntu 16.04 LTS machine draaien op adres 192.168.1.110. Nadat ik Solr geïnstalleerd heb kan ik de beheerpagina van Solr openen op http://192.168.1.110:8983. Dan zie ik het volgende:

Om een nieuwe zoekindex te maken moet er een nieuwe core in Solr worden aangemaakt. Via Core Admin kunnen we een nieuwe core aanmaken. Het pad waarin de core geplaatst wordt moet wel bestaan.








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



24

EEN SIMPEL SySTEEM VOOR BEELDHERKENNING

met Microsoft Computer Vision API en Solr

F I G U R E 2

Het makkelijkst om een basisconfiguratie aan te maken is om de configuratie uit solr-6.6.1/server/solr/configsets/basic_configs/conf te kopiëren naar de nieuwe directory starwars. Nadat Add Core succesvol is uitgevoerd en de core zichtbaar is kan via de knop core selector het schema gebouwd worden zodat we zinvol kunnen zoeken binnen Solr.

Het schema houd ik vrij simpel. Ik voeg de volgende velden toe:• woorden,typetext• titel,typestring• verschijningsdatum,dateIk gebruik type text omdat dit makkelijker een match oplevert. De case is dan niet belangrijk maar ook woorden zoals wifi geeft ook een match bij wi-fi. Titel en verschijningsdatum is natuurlijk ook belangrijk maar in dit geval niet essentieel. In dit scherm kan ik de velden via de UI van Solr aanmaken:

We willen dat alle velden in de full text zoekindex beschikbaar zijn. Dit doe je door een copy field toe te voegen. Bij source vullen we * in en als destination _text_Als alle velden zijn aangemaakt kunnen er documenten in de index van Solr gestopt worden. In dit geval gebruik ik een xml document om de informatie over filmposters te importeren.

<add>

<doc>

<field name=”id”>1</field>

<field name=”woorden”>Return of the Jedi</field>

<field name=”titel”>Return of the Jedi</field>

<field name=”verschijningsdatum”>1983-05-23T00:00:00Z</

field>

</doc>

</add>

<add>

<doc>


<field name=”woorden”>Star Wars The Empire Strikes

Back</field>

<field name=”titel”>Star Wars The Empire Strikes Back</

field>


field>

</doc>

</add>

<add>

<doc>


<field name=”woorden”>Star Wars A New Hope</field>

<field name=”titel”>Star Wars A New Hope</field>


field>

</doc>

</add>

Via documents kunnen we de import doen:

Een testje om te zoeken kan via de volgende query

http://192.168.1.110:8983/solr/starwars/select?indent=on&q=jedi&wt=json.

De output hiervan is:


CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



F I G U R E 3

F I G U R E 4

F I G U R E 6

F I G U R E 5








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



26



nn

nn

nn

nn

nn

nn

nn

26

| | | | | | | | | | | | | | | | | | | | | | |EEN SIMPEL SySTEEM VOOR BEELDHERKENNING

met Microsoft Computer Vision API en Solr

nn

nn

nn

nn

nn

nn

nn

{ “responseHeader”:{ “status”:0, “QTime”:0, “params”:{ “q”:”jedi”, “indent”:”on”, “wt”:”json”: “response”:{“numFound”:1,”start”:0,”docs”:[ { “id”:”1”, “woorden”:[“Return of the Jedi”], “titel”:”Return of the Jedi”, “verschijningsdatum”:”1983-05-23T00:00:00Z”, “_version_”:1579421629252894720}] }}

Ok dus nu hebben we iets om mee te werken. Solr kan ook zoeken op relevantie door een filter query en daarbij score als parameter op te geven. Als we de oorspronkelijk code aanvullen met:

public static async void SearchSolr(string words){

HttpClient client = new HttpClient();

string uri = “http://192.168.1.110:8983/solr/starwars/

select?fl=id,score,titel,verschijningsdatum&indent=on&q=”

+ words + “&wt=json”;

var response = await client.GetAsync(uri);

string contentString = await response.Content

.ReadAsStringAsync();

Console.WriteLine(“\nResponse:\n”);

Console.WriteLine(JsonPrettyPrint(contentString));

}

En deze code kunnen we als volgt aanroepen als we een referentie naar een ScanInfo object hebben:

// extract the words from the scan object

string wordsOnDocument = “”;

foreach (var reg in r.regions)

{

foreach (var line in reg.lines)

{

foreach (var w in line.words)

{

wordsOnDocument += w.text + “ “;

}

}

}

SearchSolr(wordsOnDocument);

De output die we dan verwachten is deze:

{“responseHeader”: {“status”: 0,“QTime”: 1,“params”: {“q”: “WAu RETUR.N OF. THE JEDI .uusfLU11iΓÇö SUZRs CAf158ER umue ΓÇóc├ªuus A Luc Am M A rox RELEASE “,“indent”: “on”,“fl”: “id,score,titel,verschijningsdatum”,“wt”: “json” }

},“response”: { “numFound”: 3, “start”: 0, “maxScore”: 3.3997817, “docs”: [{“id”: “1”,“titel”: “Return of the Jedi”,“verschijningsdatum”: “1983-05-23T00:00:00Z”,“score”: 3.3997817 },{“id”: “3”,“titel”: “Star Wars A New Hope”,“verschijningsdatum”: “1977-05-25T00:00:00Z”,“score”: 2.533875 },{“id”: “2”,“titel”: “Star Wars The Empire Strikes Back”,“verschijningsdatum”: “1980-06-20T00:00:00Z”,“score”: 0.60710394 } ] }}

Return of the Jedi is de meest relevante filmposter die we hebben kunnen vinden via een OCR van de filmposter via Cognotive Services. We hoeven alleen maar het meest waarschijnlijke te vinden. Deze voorbeelden beperkten zich tot OCR in combinatie met Solr. De Computer Vision API heeft meer zoals herkenning van objecten, persoonlijkheden, handschriftherkenning, kleurherkenning enzovoorts. Die kan je allemaal gebruiken om zoveel mogelijk relevante zaken bij Solr aan te dragen. Je hebt immers alleen de meest waarschijnlijke hit in Solr nodig.

lInks:Meer informatie over de Microsoft Cognitive Services vind je op:• https://azure.microsoft.com/nl-nl/services/cognitive-services/• https://azure.microsoft.com/nl-nl/services/cognitive-services/

computer-vision/• https://docs.microsoft.com/en-us/azure/cognitive-services/

Computer-vision/quickstarts/csharp• Voor het gebruik van Computer Vision API kun je een trial account

gebruiken: https://docs.microsoft.com/en-us/azure/cognitive-services/

Computer-vision/vision-api-how-to-topics/howtosubscribe• Solr 6.6.1 kun je hier downloaden:

http://archive.apache.org/dist/lucene/solr/• De installatie van Solr kun je hier vinden:

https://lucene.apache.org/solr/guide/6_6/installing-solr.html








FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



roel hAns bethlehem

Roel Hans Bethlehem is SharePoint en .NET architect bij de Nederlandsche Bank. Hij staat het liefst met zijn voeten in de modder en programmeert ook lekker mee met de verschillende ontwikkelteams binnen de bank. Meer informatie over mij vind je op: https://www.linkedin.com/in/rbethlehem

CLOUD

WEB

SQL

DELPHI

BIG DATA

ALM

GENERAL

Agile has a nice ring to it. Everything is agile these days. It turned agile into a hollow phrase. Unfortunately. What makes an agile project manager different then? Do his teams work in sprints? Do they organize stand-ups and retrospectives? If a company is even half-decent in agile, they will do all that already. Without an agile project manager in charge. Or did agile became part of a language full of hollow phrases where it really doesn’t matter what you stand for. According to this ad you will need to be a next generation professional. Experience in digital transformations is preferred. Eight years of experience with PRINCE2 too. You must be hip, but still know how to work Microsoft Excel. This is not different from many of the other ads I read these days. I wonder what companies hiring agile project managers, agile testers, agile developers, or even agile product owners are really looking for. To survive in this rapid changing world of technology, agile principles and practices should be everyday practice. For anyone. Since a very long time. Every developer should be an agile developer. Every project manager should already be an agile project manager. Last but not least, if you’re interested in this role and you think you quality, you need to contact the company’s vibe manager. I think I’ll pass.

First of all, as far as I know, none of the agile approaches even mention the term project management. Not extreme programming. Not Smart. Not feature driven development. And not even Scrum. Yes, I know that there is quite a lot that documents such as the Scrum guide don’t mention. And that the fact some things are left unmentioned doesn’t mean that they shouldn’t be done. But I’ve never felt a need for on-boarding traditional project management skills in agile projects. And I’ve worked in agile projects for over twenty years. The fact that organizations now, in 2017, are hiring agile project managers, makes me wonder what the state of agile is at such companies. My guess is that they are very far from what agile approaches try to establish. All agile approaches are about continuously delivering valuable software to the client. Companies hiring for agile project managers merely seem to use the term agile as branding. Whether they are hiring for agile project managers, agile developers or agile testers. ‘Although agile is the mindset according to which you act, you’re not afraid of managing PRINCE2 projects either,’ the ad states. What, in their perspective, distuingishes an agile project manager from any other subtype of project manager? Given this statement, is there any difference? Or could they have try to hire a vegan project manager or a left-handed project manager just as well? ‘Although you write with your left hand, you should not be afraid of writing with your right had either,’ the ad might as well had said.

About a week ago I noticed a personnel ad on LinkedIn from a prominent transport company in the Netherlands. Not that I’m looking for a new challenge, but this one caught my eye. The company was looking for an agile project manager. For some reason, the ad gave me the chivers. Let me explain.

Sander Hoogendoorn

The left-handed project manager









FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n










FULL-COLOUR


CONFERENCE

MAGAZINE

EVENTS

.NET

.NL

DELPHI

5 0 c y a a n1 0 0 g e e l

5 0 z w a r t1 5 c y a a n



28



nn

nn

nn

nn

nn

nn

nn

SDN MAGAZINE KNOW- LEDGE#133kunstmatige intelligentie en machine learning een grote rol. Het feit...

Documents

Transcript of SDN MAGAZINE KNOW- LEDGE#133kunstmatige intelligentie en machine learning een grote rol. Het feit...