Selene Bainum RiteTech LLC. Doing ColdFusion & SQL development for more than 1/3 of my lifetime ...
-
Upload
sabrina-casey -
Category
Documents
-
view
216 -
download
3
Transcript of Selene Bainum RiteTech LLC. Doing ColdFusion & SQL development for more than 1/3 of my lifetime ...
Selene BainumRiteTech LLC
Doing ColdFusion & SQL development for more than 1/3 of my lifetime
Chief Architect @ RiteTech RiteTech is my company – completely
independent for the first time! Likes:
SQL and databases! Love making money on the internet▪ www.smartsupplies.net
Dislikes: DC Traffic Rude People
Intro Importing Data
Tools Examples *
Exporting Data Tools Examples *
Resources
* Any custom tags/functions shown in examples are free!
SIMILARITIES
Data stored in tabular format
Both contains rows and columns
Headers/column names denote data
DIFFERENCES
Spreadsheets often contain repeated rows of data
Database tables should be normalized – repeated data among rows kept to a minimum
Spreadsheets used column delimiters – not as reliable as database columns
Spreadsheets often contain no unique key
IMPORT
Data only available as: Web pages Text files Excel spreadsheets Etc…
Need data in your database!
EXPORT
Export raw data so that others can view/manipulate it in Excel
Create reports
Bad/non-standard data Spreadsheets may take hours and
hours to clean up the data manually Ill-placed commas and quotes can
wreak havoc Data types can get misread/lost Matching imported data can be
daunting It helps to have good knowledge of SQL
Import CSV or Excel files via SQL Server
Loop over file line by line * CFHTTP * Custom Tags
cfx_text2Query cfx_Excel2Query cfx_Excel
* Problematic as CF doesn’t recognize empty string elements
Find a site that contains regularly formatted data Using list of US Representatives as an
exampleCopy the data and paste into
notepadPerform any formatting necessary
May not work on all databases Using SQL Server 2005 in example
Not all installs of databases may support this type of import
May not have access to database
Insert all rows first Start with query result set Create/modify table as needed in
database Loop over query results Insert results
Perform other database inserts/updates as needed
-- Declare the variablesDECLARE @RepID INT, @StateID INT, @DistrictID INT, @LastNameTx VARCHAR(50), @FirstNameTx VARCHAR(50),
@PhoneTx VARCHAR(25), @RoomNb INT, @StateCd VARCHAR(2), @DistrictTx VARCHAR(10)-- Create a cursorDECLARE cRep CURSOR FOR SELECT FirstName, LastName, State, District, Phone, Room FROM RepListImport WITH (NOLOCK)-- Open the cursorOPEN cRep-- Loop over the cursorFETCH NEXT FROM cRep INTO @LastNameTx, @FirstNameTx, @StateCd, @DistrictTx, @PhoneTx, @RoomNbWHILE @@FETCH_STATUS = 0 BEGIN
SET @StateID = NULL SET @DistrictID = NULL-- State exist?SELECT @StateID = StateID FROM State WITH (NOLOCK) WHERE StateCd = @StateCdIF @StateID IS NULL BEGIN
SET NOCOUNT OFFINSERT INTO State (StateCd) VALUES (@StateCd) SELECT @StateID = Scope_Identity()SET NOCOUNT ON
END-- District exist?SELECT @DistrictID = DistrictID FROM District WITH (NOLOCK) WHERE DistrictTx = @DistrictTxIF @DistrictID IS NULL BEGIN
SET NOCOUNT OFFINSERT INTO District (DistrictTx) VALUES (@DistrictTx) SELECT @DistrictID = Scope_Identity()SET NOCOUNT ON
END-- Insert the repINSERT INTO Rep (FirstNameTx, LastNameTx, PhoneTx, RoomNb, StateID, DistrictID)VALUES (@FirstNameTx, @LastNameTx, @PhoneTx, @RoomNb, @StateID, @DistrictID)-- Get the next recordFETCH NEXT FROM cRep INTO @LastNameTx, @FirstNameTx, @StateCd, @DistrictTx, @PhoneTx, @RoomNb
END-- Close the cursorCLOSE cRep DEALLOCATE cRep
Create table structures and import scripts
Schedule retrieval of files Email attachments External files
Process import when new files available Run additional database scripts
ColdFusion Database scheduler
Much easier than imports Data is already cleaned and normalized
Tricky part is formatting
Manually create a CSV file via ColdFusion
Custom Tags/Functions generateExcel cf_Excel_XML cfx_Excel cfx_Query2Excel
Manually created CSV file Created by looping over the query results Can add custom column headings Can format data in cells – date, number,
etc…generateExcel Easier than manual creation Cannot add custom column headings▪ Column heading is the same as the column name
Automatically formats numbers and dates
<cfsetting showdebugoutput="no" />
<!--- Query the reps. ---><cfquery datasource="Mayhem_SQL" name="qryGetRepList">SELECT R.RepID, R.FirstNameTx, R.LastNameTx, R.PhoneTx, R.RoomNb,
S.StateCd, D.DistrictTxFROM Preso..Rep R WITH (NOLOCK) INNER JOIN Preso..State S WITH
(NOLOCK) ON R.StateID = S.StateIDINNER JOIN Preso..District D WITH (NOLOCK) ON R.DistrictID = D.DistrictID
ORDER BY S.StateCd, D.DistrictTx</cfquery>
<cfheader name="Content-Disposition" value="inline; filename=export.csv">
<cfcontent reset="yes" type="text/comma-separated-values" />"State","District","LastName","FirstName","Phone","Room"
<cfoutput query="qryGetRepList">"#StateCd#","#DistrictTx#","#LastNameTx#","#FirstNameTx#","#PhoneTx#","#RoomNb#"
</cfoutput><cfabort />
<cfsetting showdebugoutput="no" />
<!--- Query the reps. ---><cfquery datasource="Mayhem_SQL" name="qryGetRepList">SELECT R.RepID, R.FirstNameTx, R.LastNameTx, R.PhoneTx, R.RoomNb,
S.StateCd, D.DistrictTxFROM Preso..Rep R WITH (NOLOCK) INNER JOIN Preso..State S WITH
(NOLOCK) ON R.StateID = S.StateIDINNER JOIN Preso..District D WITH (NOLOCK) ON R.DistrictID = D.DistrictID
ORDER BY S.StateCd, D.DistrictTx</cfquery>
<cfinclude template="inc_fnGenerateExcel.cfm" />
<cfscript>generateExcel(qryGetRepList, "StateCd,DistrictTx,LastNameTx,FirstNameTx,PhoneTx,RoomNb");
</cfscript><cfabort />
Creates real Excel file Uses Excel’s XML format
Custom Settings Header/column names Multiple data formatting options Multiple sheets per file
Advanced Formatting Header color/text Alternating row colors Column width
<cfsetting showdebugoutput="no" />
<!--- Query the reps. ---><cfquery datasource="Mayhem_SQL" name="qryGetRepList">SELECT R.RepID, R.FirstNameTx, R.LastNameTx, R.PhoneTx, R.RoomNb,
S.StateCd, D.DistrictTxFROM Preso..Rep R WITH (NOLOCK) INNER JOIN Preso..State S WITH
(NOLOCK) ON R.StateID = S.StateIDINNER JOIN Preso..District D WITH (NOLOCK) ON R.DistrictID = D.DistrictID
ORDER BY S.StateCd, D.DistrictTx</cfquery>
<cf_excel_xml filename="export"sheets="1"
collist1="StateCd,DistrictTx,LastNameTx,FirstNameTx,PhoneTx,RoomNb"
headerlist1="State,District,Last Name,First Name,Phone,Room" colwidths1="50,75,100,100,75,50" query1="#qryGetRepList#"> <cfabort />
<!--- Create the attribute collection. ---><cfset ac = StructNew() /> <cfset counter = 0 /><cfoutput query="qryGetRepList" group="StateCd"> <cfset counter = counter + 1 /> <cfquery dbtype="query" name="qryGetReps"> SELECT FirstNameTx, LastNameTx, PhoneTx, RoomNb, DistrictTx, StateCd FROM qryGetRepList WHERE StateCd = '#StateCd#' ORDER BY DistrictTx </cfquery> <cfoutput><cfscript> ac["collist#counter#"] =
"StateCd,DistrictTx,LastNameTx,FirstNameTx,PhoneTx,RoomNb"; ac["headerlist#counter#"] =
"State,DistrictTx,LastNameTx,FirstNameTx,PhoneTx,RoomNb"; ac["colwidths#counter#"] = "50,75,100,100,75,50"; ac["query#counter#"] = "#qryGetReps#"; ac["sheetname#counter#"] = "#StateCd#"; </cfscript></cfoutput></cfoutput>
<cf_excel_xml filename="export" sheets="#counter#" attributecollection="#ac#"> <cfabort />
Importing and Exporting can be: Easy Difficult Both
Great tools exist – use them!Search regularly for new tags & code If you know ASP.NET, even better
Adobe ColdFusion Exchange cf_Excel_XML▪ http://www.adobe.com/cfusion/exchange
Ryan Emerle cfx_text2query cfx_excel2query cfx_query2excel▪ http://www.emerle.net/old/programming/
cfTopper Generate Excel Function▪ http://www.cftopper.com/index.cfm/blogId/1/tag/
ColdFusion