IBM CMOD Composite Indexes - Revision history

Jderrick: Moved caveats section, demoted impacts.

2023-04-25T21:31:14Z

Moved caveats section, demoted impacts.

← Older revision		Revision as of 21:31, 25 April 2023
Line 17:		Line 17:

	[[File:RequiringIndexedFields.png\|800px\|Improving IBM CMOD Performance by Requiring Indexed Fields ]]		[[File:RequiringIndexedFields.png\|800px\|Improving IBM CMOD Performance by Requiring Indexed Fields ]]

	~~==Caveats==~~
	As mentioned above, it's important to use indexes on fields carefully, to ensure maximum benefit at minimum cost in terms of storage space and time. Also, the more Content Manager OnDemand Application Group fields that have indexing enabled, the more time and effort it takes to create, update, and maintain them, possibly slowing down the load process, and extending maintenance windows for arsmaint.

	= Usage =		= Usage =
Line 33:		Line 30:
	When selecting which Application Group field to add to the composite index, the order of the fields is extremely important. Use the field that has the highest cardinality (the most unique values or, alternately the fewest repeated values) generally followed by the date. This combination allows searches to identify a specific customer's documents nearly instantly, and also zero-in on a specific date range as well. Optimizing the use of Content Manager OnDemand composite indexes requires research into the types of queries that users are performing in OnDemand. Your Database Administrator responsible for IBM CMOD will have access to utilities to help determine how best to achieve better Content Manager OnDemand performance, or you can rely on the experts at [https://cmod.cloud/query-optimization/ CMOD.Cloud].		When selecting which Application Group field to add to the composite index, the order of the fields is extremely important. Use the field that has the highest cardinality (the most unique values or, alternately the fewest repeated values) generally followed by the date. This combination allows searches to identify a specific customer's documents nearly instantly, and also zero-in on a specific date range as well. Optimizing the use of Content Manager OnDemand composite indexes requires research into the types of queries that users are performing in OnDemand. Your Database Administrator responsible for IBM CMOD will have access to utilities to help determine how best to achieve better Content Manager OnDemand performance, or you can rely on the experts at [https://cmod.cloud/query-optimization/ CMOD.Cloud].

	= Impact on Storage =		=Caveats=
			As mentioned above, it's important to use indexes on fields carefully, to ensure maximum benefit at minimum cost in terms of storage space, bandwidth, and time. Also, the more Content Manager OnDemand Application Group fields that have indexing enabled, the more time and effort it takes to create, update, and maintain them, possibly slowing down the load process, and extending maintenance windows for arsmaint.

			== Impact on Storage ==
	Composite indexes will require that you store the contents of both columns, along with a list of database page identifiers. If your CMOD Application Group is especially large, this can cause the database tables for that specific App Group to consume anywhere from 10 to 50% more space. Check with your Database Administrators and System Administrators to ensure that you have enough space to complete the index creation.		Composite indexes will require that you store the contents of both columns, along with a list of database page identifiers. If your CMOD Application Group is especially large, this can cause the database tables for that specific App Group to consume anywhere from 10 to 50% more space. Check with your Database Administrators and System Administrators to ensure that you have enough space to complete the index creation.

	= Impact on I/O =		== Impact on I/O ==
	You may think to yourself, "Storage is cheap, let's index everything!" but there's different cost -- storage bandwidth. In order for indexes to be the most effective, indexes must be '''compact''' - the fewest number of rows in the index with a relatively short list of records following after it. Building Multi Field Indexes in CMOD with a large number of fields could effectively DOUBLE the size of the database table -- making the task of searching it just as slow as not having an index at all.		You may think to yourself, "Storage is cheap, let's index everything!" but there's different cost -- storage bandwidth. In order for indexes to be the most effective, indexes must be '''compact''' - the fewest number of rows in the index with a relatively short list of records following after it. Building Multi Field Indexes in CMOD with a large number of fields could effectively DOUBLE the size of the database table -- making the task of searching it just as slow as not having an index at all.

Jderrick: Several updates and added section on CMOD Multiple Field Index cost in terms of I/O bandwidth.

2023-04-25T20:11:46Z

Several updates and added section on CMOD Multiple Field Index cost in terms of I/O bandwidth.

← Older revision		Revision as of 20:11, 25 April 2023
Line 1:		Line 1:
	{{TOCright}}		{{TOCright}}
	= Disambiguation =		= Disambiguation =
			Also known as "Content Manager OnDemand Multi Field Indexes".

	You might be looking for information on getting metadata from documents to be loaded into Content Manager OnDemand Applications using ACIF - see the IBM CMOD Indexing Reference: [https://www.ibm.com/support/knowledgecenter/SSEPCD_10.5.0/com.ibm.ondemand.ir.doc/dodnt000.htm v10.5]   [https://cmod.wiki/dox/CMODv10.1/IndexingReference.pdf v10.1]   [https://cmod.wiki/dox/CMODv9.5/IndexingReference.pdf v9.5]   [https://cmod.wiki/dox/CMODv9.0/IndexingReference.pdf v9.0]   [https://cmod.wiki/dox/CMODv8.5/Indexing_Reference.pdf v8.5]   [https://cmod.wiki/dox/CMODv8.4/Indexing_Reference.pdf v8.4]		You might be looking for information on getting metadata from documents to be loaded into Content Manager OnDemand Applications using ACIF - see the IBM CMOD Indexing Reference: [https://www.ibm.com/support/knowledgecenter/SSEPCD_10.5.0/com.ibm.ondemand.ir.doc/dodnt000.htm v10.5]   [https://cmod.wiki/dox/CMODv10.1/IndexingReference.pdf v10.1]   [https://cmod.wiki/dox/CMODv9.5/IndexingReference.pdf v9.5]   [https://cmod.wiki/dox/CMODv9.0/IndexingReference.pdf v9.0]   [https://cmod.wiki/dox/CMODv8.5/Indexing_Reference.pdf v8.5]   [https://cmod.wiki/dox/CMODv8.4/Indexing_Reference.pdf v8.4]

Line 6:		Line 8:
	In IBM Content Manager OnDemand, an Application Group's field type can be set to Filter or Index. Setting the type to 'Index' tells CMOD that it should instruct the database to build an index for that field. Setting the value to Filter means that the field is used to filter the results after the index search is completed.		In IBM Content Manager OnDemand, an Application Group's field type can be set to Filter or Index. Setting the type to 'Index' tells CMOD that it should instruct the database to build an index for that field. Setting the value to Filter means that the field is used to filter the results after the index search is completed.

	Database indexes make access to data faster by storing a ~~value with~~ the location of the data in the index -- similar to how the index at the back of a reference book works. The advantage is that if the data being indexed is mostly unique ("high cardinality") the indexes can be used to make searches very fast. In the OnDemand Application Group configuration window, in the Field Information tab, fields that are set to 'Index' should also have the 'Required' checkbox selected in the Folder configuration, Field Information Tab. This forces end users to enter a value into the field in order to perform a search - reducing the potential for end users to submit queries that will perform poorly (consuming a disproportionate amount of server I/O bandwidth, RAM, and CPU), as well as reducing the possibility that one of these 'rogue queries' will impact the performance of other user's searches or retrievals.		Database indexes make access to data faster by storing a list of metadata stored inside the database table alongside the location of the data in the index -- similar to how the index at the back of a reference book works. The advantage is that if the data being indexed is mostly unique ("high cardinality" is the term to use when talking to your Database Administrator) the indexes can be used to make searches very fast. In the OnDemand Application Group configuration window, in the Field Information tab, fields that are set to 'Index' should also have the 'Required' checkbox selected in the Folder configuration, Field Information Tab. This forces end users to enter a value into the field in order to perform a search - reducing the potential for end users to submit queries that will perform poorly (consuming a disproportionate amount of server I/O bandwidth, RAM, and CPU), as well as reducing the possibility that one of these 'rogue queries' will impact the performance of other user's searches or retrievals.

	For those who like analogies: Using 'Filter' Field Type is like trying to find a word by searching every page in a very long book. Using 'Index' is like using the index at the back of a book, telling you where to find each occurrence of the word by page number. In fact, this naming convention is used in databases -- indexes point to database pages (like pages in a book), which contain records ("words"). For this reason, you want indexes on fields that are the '''most unique''' like 'customer number' which should uniquely identify a customer account, but not 'zip code', which could encompass tens of thousands of customers in large cities. To continue the book analogy, the index in the back of your book doesn't contain a list of the pages that the words "and", "the", "a" or "to" -- because those words are likely to appear on every page of the book, and are not just useless, but also consume a huge amount of space while being useless.		For those who like analogies: Using 'Filter' Field Type is like trying to find a word by searching every page in a very long book. Using 'Index' is like using the index at the back of a book, telling you where to find each occurrence of the word by page number. In fact, this naming convention is used in databases -- indexes point to database pages (like pages in a book), which contain records ("words"). For this reason, you want indexes on fields that are the '''most unique''' like 'customer number' which should uniquely identify a customer account, but not 'zip code', which could encompass tens of thousands of customers in large cities. To continue the book analogy, the index in the back of your book doesn't contain a list of the pages that the words "and", "the", "a" or "to" -- because those words are likely to appear on every page of the book, and are not just useless, but also consume a huge amount of space while being useless.

	Composite Indexes are database indexes that are created by adding the values of two or more fields together. If the fields selected for the composite indexes are mostly unique (high specificity or cardinality) the index may be very large, but it should be able to identify a single record -- and be extraordinarily efficient in the process. In order to be effective at reducing the duration of queries, ALL of the fields that appear in the composite index must also be set to be 'Required' fields in the Folder configuration.		Composite Indexes are database indexes that are created by adding the values of two or more fields together. If the fields selected for the composite indexes are mostly unique (high specificity or cardinality) the index may be very large, but it should be able to identify a single record -- and be extraordinarily efficient in the process. In order to be effective at reducing the duration of queries, ALL of the fields that appear in the composite index must also be set to be 'Required' fields in the Folder configuration.

			Finally the '''order''' that the fields appear in the OnDemand Multiple Field Index appears in is also important - the highest cardinality ("most unique") field must appear first for maximum performance.

	[[File:RequiringIndexedFields.png\|800px\|Improving IBM CMOD Performance by Requiring Indexed Fields ]]		[[File:RequiringIndexedFields.png\|800px\|Improving IBM CMOD Performance by Requiring Indexed Fields ]]
Line 31:		Line 35:
	= Impact on Storage =		= Impact on Storage =
	Composite indexes will require that you store the contents of both columns, along with a list of database page identifiers. If your CMOD Application Group is especially large, this can cause the database tables for that specific App Group to consume anywhere from 10 to 50% more space. Check with your Database Administrators and System Administrators to ensure that you have enough space to complete the index creation.		Composite indexes will require that you store the contents of both columns, along with a list of database page identifiers. If your CMOD Application Group is especially large, this can cause the database tables for that specific App Group to consume anywhere from 10 to 50% more space. Check with your Database Administrators and System Administrators to ensure that you have enough space to complete the index creation.

			= Impact on I/O =
			You may think to yourself, "Storage is cheap, let's index everything!" but there's different cost -- storage bandwidth. In order for indexes to be the most effective, indexes must be '''compact''' - the fewest number of rows in the index with a relatively short list of records following after it. Building Multi Field Indexes in CMOD with a large number of fields could effectively DOUBLE the size of the database table -- making the task of searching it just as slow as not having an index at all.

Jderrick: Added more analogies about CMOD index fields, and caveats for choosing which fields should be CMOD indexes.

2020-11-20T22:30:15Z

Added more analogies about CMOD index fields, and caveats for choosing which fields should be CMOD indexes.

← Older revision		Revision as of 22:30, 20 November 2020
Line 8:		Line 8:
	Database indexes make access to data faster by storing a value with the location of the data in the index -- similar to how the index at the back of a reference book works. The advantage is that if the data being indexed is mostly unique ("high cardinality") the indexes can be used to make searches very fast. In the OnDemand Application Group configuration window, in the Field Information tab, fields that are set to 'Index' should also have the 'Required' checkbox selected in the Folder configuration, Field Information Tab. This forces end users to enter a value into the field in order to perform a search - reducing the potential for end users to submit queries that will perform poorly (consuming a disproportionate amount of server I/O bandwidth, RAM, and CPU), as well as reducing the possibility that one of these 'rogue queries' will impact the performance of other user's searches or retrievals.		Database indexes make access to data faster by storing a value with the location of the data in the index -- similar to how the index at the back of a reference book works. The advantage is that if the data being indexed is mostly unique ("high cardinality") the indexes can be used to make searches very fast. In the OnDemand Application Group configuration window, in the Field Information tab, fields that are set to 'Index' should also have the 'Required' checkbox selected in the Folder configuration, Field Information Tab. This forces end users to enter a value into the field in order to perform a search - reducing the potential for end users to submit queries that will perform poorly (consuming a disproportionate amount of server I/O bandwidth, RAM, and CPU), as well as reducing the possibility that one of these 'rogue queries' will impact the performance of other user's searches or retrievals.

	For those who like analogies: Using 'Filter' Field Type is like trying to find a word by searching every page in a very long book. Using 'Index' is like using the index at the back of a book, telling you where to find each occurrence of the word by page number. In fact, this naming convention is used in databases -- indexes point to database pages (like pages in a book), which contain records ("words").		For those who like analogies: Using 'Filter' Field Type is like trying to find a word by searching every page in a very long book. Using 'Index' is like using the index at the back of a book, telling you where to find each occurrence of the word by page number. In fact, this naming convention is used in databases -- indexes point to database pages (like pages in a book), which contain records ("words"). For this reason, you want indexes on fields that are the '''most unique''' like 'customer number' which should uniquely identify a customer account, but not 'zip code', which could encompass tens of thousands of customers in large cities. To continue the book analogy, the index in the back of your book doesn't contain a list of the pages that the words "and", "the", "a" or "to" -- because those words are likely to appear on every page of the book, and are not just useless, but also consume a huge amount of space while being useless.

	Composite Indexes are database indexes that are created by adding the values of two or more fields together. If the fields selected for the composite indexes are mostly unique (high specificity or cardinality) the index may be very large, but it should be able to identify a single record -- and be extraordinarily efficient in the process. In order to be effective at reducing the duration of queries, ALL of the fields that appear in the composite index must also be set to be 'Required' fields in the Folder configuration.		Composite Indexes are database indexes that are created by adding the values of two or more fields together. If the fields selected for the composite indexes are mostly unique (high specificity or cardinality) the index may be very large, but it should be able to identify a single record -- and be extraordinarily efficient in the process. In order to be effective at reducing the duration of queries, ALL of the fields that appear in the composite index must also be set to be 'Required' fields in the Folder configuration.

	[[File:RequiringIndexedFields.png\|800px\|Improving IBM CMOD Performance by Requiring Indexed Fields ]]		[[File:RequiringIndexedFields.png\|800px\|Improving IBM CMOD Performance by Requiring Indexed Fields ]]

			==Caveats==
			As mentioned above, it's important to use indexes on fields carefully, to ensure maximum benefit at minimum cost in terms of storage space and time. Also, the more Content Manager OnDemand Application Group fields that have indexing enabled, the more time and effort it takes to create, update, and maintain them, possibly slowing down the load process, and extending maintenance windows for arsmaint.

	= Usage =		= Usage =

Jderrick: Changed image size for CMOD Indexed Fields graphic.

2020-10-21T14:42:00Z

Changed image size for CMOD Indexed Fields graphic.

← Older revision		Revision as of 14:42, 21 October 2020
Line 12:		Line 12:
	Composite Indexes are database indexes that are created by adding the values of two or more fields together. If the fields selected for the composite indexes are mostly unique (high specificity or cardinality) the index may be very large, but it should be able to identify a single record -- and be extraordinarily efficient in the process. In order to be effective at reducing the duration of queries, ALL of the fields that appear in the composite index must also be set to be 'Required' fields in the Folder configuration.		Composite Indexes are database indexes that are created by adding the values of two or more fields together. If the fields selected for the composite indexes are mostly unique (high specificity or cardinality) the index may be very large, but it should be able to identify a single record -- and be extraordinarily efficient in the process. In order to be effective at reducing the duration of queries, ALL of the fields that appear in the composite index must also be set to be 'Required' fields in the Folder configuration.

	[[File:RequiringIndexedFields.png\|Improving IBM CMOD Performance by Requiring Indexed Fields ]]		[[File:RequiringIndexedFields.png\|800px\|Improving IBM CMOD Performance by Requiring Indexed Fields ]]

	= Usage =		= Usage =

Jderrick: One too many semicolons in CMOD v10.5 link.

2020-09-25T14:54:02Z

One too many semicolons in CMOD v10.5 link.

← Older revision		Revision as of 14:54, 25 September 2020
Line 1:		Line 1:
	{{TOCright}}		{{TOCright}}
	= Disambiguation =		= Disambiguation =
	You might be looking for information on getting metadata from documents to be loaded into Content Manager OnDemand Applications using ACIF - see the IBM CMOD Indexing Reference: [https://www.ibm.com/support/knowledgecenter/SSEPCD_10.5.0/com.ibm.ondemand.ir.doc/dodnt000.htm v10.5]  ; [https://cmod.wiki/dox/CMODv10.1/IndexingReference.pdf v10.1]   [https://cmod.wiki/dox/CMODv9.5/IndexingReference.pdf v9.5]   [https://cmod.wiki/dox/CMODv9.0/IndexingReference.pdf v9.0]   [https://cmod.wiki/dox/CMODv8.5/Indexing_Reference.pdf v8.5]   [https://cmod.wiki/dox/CMODv8.4/Indexing_Reference.pdf v8.4]		You might be looking for information on getting metadata from documents to be loaded into Content Manager OnDemand Applications using ACIF - see the IBM CMOD Indexing Reference: [https://www.ibm.com/support/knowledgecenter/SSEPCD_10.5.0/com.ibm.ondemand.ir.doc/dodnt000.htm v10.5]   [https://cmod.wiki/dox/CMODv10.1/IndexingReference.pdf v10.1]   [https://cmod.wiki/dox/CMODv9.5/IndexingReference.pdf v9.5]   [https://cmod.wiki/dox/CMODv9.0/IndexingReference.pdf v9.0]   [https://cmod.wiki/dox/CMODv8.5/Indexing_Reference.pdf v8.5]   [https://cmod.wiki/dox/CMODv8.4/Indexing_Reference.pdf v8.4]

	= Background =		= Background =

Jderrick: Added link to IBM CMOD v10.5

2020-09-25T14:53:21Z

Added link to IBM CMOD v10.5

← Older revision		Revision as of 14:53, 25 September 2020
Line 1:		Line 1:
	{{TOCright}}		{{TOCright}}
	= Disambiguation =		= Disambiguation =
	You might be looking for information on getting metadata from documents to be loaded into Content Manager OnDemand Applications using ACIF - see the IBM CMOD Indexing Reference: [https://cmod.wiki/dox/CMODv10.1/IndexingReference.pdf v10.1]   [https://cmod.wiki/dox/CMODv9.5/IndexingReference.pdf v9.5]   [https://cmod.wiki/dox/CMODv9.0/IndexingReference.pdf v9.0]   [https://cmod.wiki/dox/CMODv8.5/Indexing_Reference.pdf v8.5]   [https://cmod.wiki/dox/CMODv8.4/Indexing_Reference.pdf v8.4]		You might be looking for information on getting metadata from documents to be loaded into Content Manager OnDemand Applications using ACIF - see the IBM CMOD Indexing Reference: [https://www.ibm.com/support/knowledgecenter/SSEPCD_10.5.0/com.ibm.ondemand.ir.doc/dodnt000.htm v10.5]  ; [https://cmod.wiki/dox/CMODv10.1/IndexingReference.pdf v10.1]   [https://cmod.wiki/dox/CMODv9.5/IndexingReference.pdf v9.5]   [https://cmod.wiki/dox/CMODv9.0/IndexingReference.pdf v9.0]   [https://cmod.wiki/dox/CMODv8.5/Indexing_Reference.pdf v8.5]   [https://cmod.wiki/dox/CMODv8.4/Indexing_Reference.pdf v8.4]

	= Background =		= Background =

Jderrick: Lots of generic minor updates.

2020-03-21T02:08:51Z

Lots of generic minor updates.

← Older revision		Revision as of 02:08, 21 March 2020
Line 6:		Line 6:
	In IBM Content Manager OnDemand, an Application Group's field type can be set to Filter or Index. Setting the type to 'Index' tells CMOD that it should instruct the database to build an index for that field. Setting the value to Filter means that the field is used to filter the results after the index search is completed.		In IBM Content Manager OnDemand, an Application Group's field type can be set to Filter or Index. Setting the type to 'Index' tells CMOD that it should instruct the database to build an index for that field. Setting the value to Filter means that the field is used to filter the results after the index search is completed.

	Database indexes make access to data faster by storing a value with the location of the data in the index -- similar to how the index at the back of a reference book works. The advantage is that if the data being indexed is mostly unique ("high cardinality") the indexes can be used to make searches very fast. In OnDemand, fields that are set to 'Index' should also have the 'Required' checkbox selected in the Folder configuration, Field Information Tab.		Database indexes make access to data faster by storing a value with the location of the data in the index -- similar to how the index at the back of a reference book works. The advantage is that if the data being indexed is mostly unique ("high cardinality") the indexes can be used to make searches very fast. In the OnDemand Application Group configuration window, in the Field Information tab, fields that are set to 'Index' should also have the 'Required' checkbox selected in the Folder configuration, Field Information Tab. This forces end users to enter a value into the field in order to perform a search - reducing the potential for end users to submit queries that will perform poorly (consuming a disproportionate amount of server I/O bandwidth, RAM, and CPU), as well as reducing the possibility that one of these 'rogue queries' will impact the performance of other user's searches or retrievals.

	For those who like analogies: Using 'Filter' Field Type is like trying to find a word by searching every page in a very long book. Using 'Index' is like using the ~~Index~~ at the back of a book, telling you where to find each occurrence of the word by page number. In fact, this naming convention is used in databases -- ~~index~~ point to database pages, which contain records ("words").		For those who like analogies: Using 'Filter' Field Type is like trying to find a word by searching every page in a very long book. Using 'Index' is like using the index at the back of a book, telling you where to find each occurrence of the word by page number. In fact, this naming convention is used in databases -- indexes point to database pages (like pages in a book), which contain records ("words").

	Composite Indexes are database indexes that are created by adding the values of two or more fields together. If the fields selected for the composite indexes are mostly unique (high specificity or cardinality) the index may be very large, but it should be able to identify a single record -- and be extraordinarily efficient in the process. In order to be effective at reducing the duration of queries, ALL of the fields that appear in the composite index must also be set to be 'Required' fields in the Folder configuration.		Composite Indexes are database indexes that are created by adding the values of two or more fields together. If the fields selected for the composite indexes are mostly unique (high specificity or cardinality) the index may be very large, but it should be able to identify a single record -- and be extraordinarily efficient in the process. In order to be effective at reducing the duration of queries, ALL of the fields that appear in the composite index must also be set to be 'Required' fields in the Folder configuration.
Line 16:		Line 16:
	= Usage =		= Usage =

	In solutions where OnDemand must process a large number of queries in a short period of time (~~for example,~~ tax ~~preparation~~ season) creating indexes on a few single fields might not be enough to ensure that all queries are completed quickly. In these situations, IBM CMOD can be configured to have a 'composite index' - a single index which contains more than one field - like CUST_NUM+STMT_DATE to combine 'customer number' and 'statement date' data into a single index record. This means that a customer who searches for their customer number for a particular date should be able to find their statement immediately, because the index tells the database where to find the rest of their information in the database table.		In solutions where Content Manager OnDemand must process a large number of queries in a short period of time (like tax season) creating indexes on a few single fields might not be enough to ensure that all queries are completed quickly. In these situations, IBM CMOD can be configured to have a 'composite index' - a single index which contains more than one field - like CUST_NUM+STMT_DATE to combine 'customer number' and 'statement date' data into a single index record. This means that a customer who searches for their customer number for a particular date should be able to find their statement immediately, because the index tells the database where to find the rest of their information in the database table.

			Composite indexes are especially effective when building line-of-business front-ends through the ODWEK Java API, where you have strict control over the fields used to search for specific documents, like investment statements.

	Image: Advanced Indexing Parameters window & configured composite index.		Image: Advanced Indexing Parameters window & configured composite index.
Line 22:		Line 24:
	= Selecting fields =		= Selecting fields =

	When selecting which Application Group field to add to the composite index, the order of the fields is extremely important. Use the field that has the highest cardinality (the most unique values or the fewest repeated values) generally followed by the date. This combination allows searches to identify documents ~~quickly~~, and also zero-in on a specific date range as well. Optimizing the use of Content Manager OnDemand composite indexes requires research into the types of queries that users are performing in OnDemand. Your Database ~~Adminstrator~~ responsible for IBM CMOD will have access to utilities to help determine how best to achieve better Content Manager OnDemand performance.		When selecting which Application Group field to add to the composite index, the order of the fields is extremely important. Use the field that has the highest cardinality (the most unique values or, alternately the fewest repeated values) generally followed by the date. This combination allows searches to identify a specific customer's documents nearly instantly, and also zero-in on a specific date range as well. Optimizing the use of Content Manager OnDemand composite indexes requires research into the types of queries that users are performing in OnDemand. Your Database Administrator responsible for IBM CMOD will have access to utilities to help determine how best to achieve better Content Manager OnDemand performance, or you can rely on the experts at [https://cmod.cloud/query-optimization/ CMOD.Cloud].

	= Impact on Storage =		= Impact on Storage =
	Composite indexes will require that you store the contents of both columns, along with a list of database page identifiers. If your ~~IBM~~ CMOD Application Group is especially large, this can cause the database to consume anywhere from 10 to 50% more space. Check with your Database Administrators and System Administrators to ensure that you have enough space to complete the index creation.		Composite indexes will require that you store the contents of both columns, along with a list of database page identifiers. If your CMOD Application Group is especially large, this can cause the database tables for that specific App Group to consume anywhere from 10 to 50% more space. Check with your Database Administrators and System Administrators to ensure that you have enough space to complete the index creation.

Jderrick: Corrected typos, added section on IBM CMOD query.

2018-08-22T16:13:45Z

Corrected typos, added section on IBM CMOD query.

← Older revision		Revision as of 16:13, 22 August 2018
Line 22:		Line 22:
	= Selecting fields =		= Selecting fields =

	When selecting which Application Group field to add to the composite index, the order of the fields is extremely important. Use the field that has the highest ~~cardinarlity~~ (the most unique values) generally followed by the date. This combination allows searches to identify documents quickly, and also zero-in on a specific date range as well.		When selecting which Application Group field to add to the composite index, the order of the fields is extremely important. Use the field that has the highest cardinality (the most unique values or the fewest repeated values) generally followed by the date. This combination allows searches to identify documents quickly, and also zero-in on a specific date range as well. Optimizing the use of Content Manager OnDemand composite indexes requires research into the types of queries that users are performing in OnDemand. Your Database Adminstrator responsible for IBM CMOD will have access to utilities to help determine how best to achieve better Content Manager OnDemand performance.

	= Impact on Storage =		= Impact on Storage =
	Composite indexes will require that you store the contents of both columns, along with a list of database page identifiers. If your IBM CMOD Application Group is especially large, this can cause the database to consume anywhere from 10 to 50% more space. Check with your Database Administrators and System Administrators to ensure that you have enough space to complete the index creation.		Composite indexes will require that you store the contents of both columns, along with a list of database page identifiers. If your IBM CMOD Application Group is especially large, this can cause the database to consume anywhere from 10 to 50% more space. Check with your Database Administrators and System Administrators to ensure that you have enough space to complete the index creation.

Jderrick: Added IBM CMOD Performance image, corrected spelling mistakes.

2018-08-22T16:09:06Z

Added IBM CMOD Performance image, corrected spelling mistakes.

← Older revision		Revision as of 16:09, 22 August 2018
Line 6:		Line 6:
	In IBM Content Manager OnDemand, an Application Group's field type can be set to Filter or Index. Setting the type to 'Index' tells CMOD that it should instruct the database to build an index for that field. Setting the value to Filter means that the field is used to filter the results after the index search is completed.		In IBM Content Manager OnDemand, an Application Group's field type can be set to Filter or Index. Setting the type to 'Index' tells CMOD that it should instruct the database to build an index for that field. Setting the value to Filter means that the field is used to filter the results after the index search is completed.

	Database indexes make access to data faster by storing a value with the location of the data in the index -- similar to how the index in a book works. The advantage is that if the data being indexed is mostly unique ("high cardinality") the indexes can be used to make searches very fast. In OnDemand, fields that are set to 'Index' should also have the 'Required' checkbox selected in the Folder configuration, Field Information Tab.		Database indexes make access to data faster by storing a value with the location of the data in the index -- similar to how the index at the back of a reference book works. The advantage is that if the data being indexed is mostly unique ("high cardinality") the indexes can be used to make searches very fast. In OnDemand, fields that are set to 'Index' should also have the 'Required' checkbox selected in the Folder configuration, Field Information Tab.

	For those who like analogies: Using 'Filter' Field Type is like trying to find a word by searching every page in a very long book. Using 'Index' is like using the Index at the back of a book, telling you where to find each ~~occurence~~ of the word by page number. In fact, this naming convention is used in databases -- index point to database pages, which contain records ("words").		For those who like analogies: Using 'Filter' Field Type is like trying to find a word by searching every page in a very long book. Using 'Index' is like using the Index at the back of a book, telling you where to find each occurrence of the word by page number. In fact, this naming convention is used in databases -- index point to database pages, which contain records ("words").

	Composite Indexes are database indexes that are created by adding the values of two or more fields together. If the fields selected for the composite indexes are mostly unique (high specificity or cardinality) the index may be very large, but it should be able to identify a single record -- and be extraordinarily efficient in the process. In order to be effective at reducing the duration of queries, ALL of the fields that appear in the composite index must also be set to be 'Required' fields in the Folder configuration.		Composite Indexes are database indexes that are created by adding the values of two or more fields together. If the fields selected for the composite indexes are mostly unique (high specificity or cardinality) the index may be very large, but it should be able to identify a single record -- and be extraordinarily efficient in the process. In order to be effective at reducing the duration of queries, ALL of the fields that appear in the composite index must also be set to be 'Required' fields in the Folder configuration.

	~~Image~~: ~~Application Group Field Type Index & Folder Field Information Required Selected~~		[[File:RequiringIndexedFields.png\|Improving IBM CMOD Performance by Requiring Indexed Fields ]]

	= Usage =		= Usage =

Jderrick: Tweaks to IBM CMOD Composite Indexes

2018-08-10T17:32:54Z

Tweaks to IBM CMOD Composite Indexes

← Older revision		Revision as of 17:32, 10 August 2018
Line 1:		Line 1:
	~~IBM CMOD Composite Index Guide~~		{{TOCright}}
	= Disambiguation =		= Disambiguation =
	;IBM CMOD Indexing Reference ~~- a guide for using ACIF to collect metadata from documents~~		You might be looking for information on getting metadata from documents to be loaded into Content Manager OnDemand Applications using ACIF - see the IBM CMOD Indexing Reference: [https://cmod.wiki/dox/CMODv10.1/IndexingReference.pdf v10.1]   [https://cmod.wiki/dox/CMODv9.5/IndexingReference.pdf v9.5]   [https://cmod.wiki/dox/CMODv9.0/IndexingReference.pdf v9.0]   [https://cmod.wiki/dox/CMODv8.5/Indexing_Reference.pdf v8.5]   [https://cmod.wiki/dox/CMODv8.4/Indexing_Reference.pdf v8.4]
	:[https://cmod.wiki/dox/CMODv10.1/IndexingReference.pdf v10.1]
	:[https://cmod.wiki/dox/CMODv9.5/IndexingReference.pdf v9.5]
	:[https://cmod.wiki/dox/CMODv9.0/IndexingReference.pdf v9.0]
	:[https://cmod.wiki/dox/CMODv8.5/Indexing_Reference.pdf v8.5]
	:[https://cmod.wiki/dox/CMODv8.4/Indexing_Reference.pdf v8.4]

	= Background =		= Background =