Gold Sponsors Bronze SponsorsSilver Sponsors

Large List SupportList ThrottlingRick Rosato & Benjamin Wong

Agenda

• Large List support • What this means? (Throttling/Threshold/Limits)• Strategies for managing large lists in SharePoint

• Not covering • Office 365• Amount of content • Size considerations • Content Type management • Archiving • Storage strategies • If you have any of these issue, come see me

afterwards!

What usually happens?

• Using SharePoint happily • Upload or add to the list• Listview throws an error, you have

insufficient privileges to view the page content. • You panic, your world is imploding• You call the helpdesk• They scramble to solve the issue

SharePoint 2013 Limits

• Lists and Libraries – limits (SharePoint 2013) • 50 million (items) • 2 GB (item or attachment)

• Threshold Limit is 5,000, why 5,000? • Minimize database contention (row-level

locking)

• When we do get errors, what’s going to happen?

These things stop if I exceed the threshold?

• List issues • Datasheet view breaks -

permission error • Add/remove/update a list

column/content type• Create/remove an index• Manage files which have

no checked-in version• Non-indexed recursive

queries• Cross list query• Lookup columns that

enforce relationship behaviour

• Deleting a list/site

• Save list as template with data

• Showing totals in list views

• Enable/disable attachments in a list

• Folder issues • Delete/copy/rename• Queries that filter on non-

indexed col • Setting item-level

security • Open with Explorer

Behind the scenes

Resource throttles and limits

Threshold or Limit Default value Description

List View Threshold 5,000 (warning at 3,000)

Specifies the maximum number of list or library items that a database operation, such as a query, can process at one time. Operations that exceed this limit are blocked.

Unique permissions limit 50,000 (breaking the inheritance is counted as 1 unique permission)

Specifies the maximum number of unique permissions allowed for a list or library.

Row size limit per list (internal rows)

6 (8,000 bytes)(related to column

Specifies the maximum number of table rows internal to the database used for a list or library item. To accommodate wide lists with many columns, each item is wrapped over several internal table rows,.

Continue…

Resource throttles and limits (cont..)

Threshold or Limit Default value Description

List View Lookup Threshold 12 (joins) Specifies the maximum number of join operations, such as those based on lookup, Person/Group, or workflow status columns.

List View Threshold size for auditors and administrators

20,000 Specifies the maximum number of list or library items that a database operation, such as a query, can process at one time when performed by an auditor or administrator with appropriate permissions. This setting works in conjunction with Allow Object Model Override.

Allow Object Model Override

Y Specifies whether or not developers can perform database operations, such as queries, that request an override of the List View Threshold to the higher limit specified by the List View Threshold for auditors and administrators.

Daily time window None Specifies a time period during which resource thresholds and limits are ignored. An administrator can configure a time window during "off-peak" hours in 15 minute increments and up to 24 hours, such as, 6:00 PM to 10:00 PM or 1:30 AM to 5:15 AM.

Ways to manage large lists and libraries

• Create SharePoint Indexed Columns • Create Filtered Views Based on Column

Indexes • Organizing Items into Folders • Controlling the display of data on the page• Using offline synchronisation • Using Search to find list or library items • Using indexed and fallback queries • Using External Data

Create SharePoint indexed Columns

• Help improve performance, up to 20 columns • Only add to those you need, does add

overhead Supported Column Types Unsupported Column Types

Single line of text Multiple lines of text

Choice (single value) Choice (multi-valued)

Number Calculated

Currency Hyperlink or Picture

Date and Time Custom Columns

Lookup (single value) Lookup (multi-valued)

Person or Group (single value) Person or Group (multi-valued)

Yes/No External data External data

Managed Metadata

Create Filtered Views Based on Column Indexes

• First filter MUST be indexed (to return less than List View Threshold)• Other filtered columns not countered (except for

devs using compound indexes) • Multi-filter with AND operator to limit results • Recycle Bin may skew results set • Some operations might still be blocked (requires

entire list)• adding or deleting an index• creating a sort in the view definition;• displaying a column total• adding, updating, or deleting calculated fields.

Organising Items into Folders• Folders - are an internal index• Subfolders count as an item (but not

items in that subfolder). • Good quick strategy for fixing List View

Threshold issues.• Moving between folders is difficult

Controlling the display of data on the page

• Setting an item limit on a page• Display items in batches of the specified size

(for paging), • Limit the total number of items returned to the

specified amount

• Using the Preview Pane view style• Using the Group By view style• Using RSS (turn on Site Collection and List)

• Configure List RSS - Based on modified date – limit results

• Set the maximum number of days to include

Using offline synchronisation• Access 2013 • Local copy, synchronises when on online

(upto 2GB) • 50,000 items (configurable)

• Outlook 2013 • Contacts• Tasks • Calendar• Discussion List • Documents • Creates a PST file, synchronises when on

online.

Using Search to find list or library items

• It own indexing mechanisms, not subject to the List View Threshold• Results can be scoped narrow or widen

as needed• Search is really fast when querying the

list/library • Search on list/library attributes and use

refiners to limit results

Metadata Navigation Feature and using indexed and fallback queries

• Metadata Navigation and Filtering can assist with large lists, via query optimisation and best selection of queries.• If there’s no best query, Metadata Nav &

Filtering will construct and execute a fallback query – limited results returned. • Intended to return some useful results

when original query is blocked due to large list throttling.• Not 100% reliable

Using External Data

• Q. Should it be in SharePoint in the first place? • Using BCS – provides standardized

interfaces to external data sources• External List usually relates to

transactional data – LoB, ERP. • Not bound be list throttling limits, it has

it’s own configuration.

Demos

• Create a simple or compound index • Create a filtered view based on indexed

column• Metadata navigation • Central Admin - Resource Throttling

• Qualitem Solution • Large List administration • Content Rollup support for large list • Paging and Search

Don’ts

• Lookup columns, don’t exceed 12 per list • What classifies as a Lookup column?

• Name (linked to document/edit with menu) • Edit (link to edit item) • Type (icon linked to document)• Standard lookup columns• Managed metadata (single and multiple-value)• People and group columns (single and

multiple-value people and group columns• Created by , Modified by ( people ) • Workflow Status

If you get stuck!

• Buy our tools to help you manage your lists – Qualitem • Increase the List View Threshold

temporarily via CA • Administrators can run queries on large

lists• Enable a daily time window for large

queries in CA

• Call me – Rick Rosato – 04 1111 7270 (rick@qualitem.com)

Reference

• http://office.microsoft.com/en-in/sharepoint-server-help/manage-lists-and-libraries-with-many-items-HA102771361.aspx#_Toc346115854• http://technet.microsoft.com/en-us/library

/cc262813(v=office.14).aspx

• Rick Rosato (rick@qualitem.com)

& A