InsightIQ can export two kinds of statistics:
- cluster performance statistics
using the command iiq_data_export perf export - file system analytic’s
using the command: iiq_data_export fsa export
In this exercise we will focus on FSA statistics
Note: the following commands where tested on InsightIQ 3.1
iiq_data_export fsa list
iiq_data export fsa list usage/switches:
iiq_data_export fsa list {–clusters | –data-modules | –breakouts | –filters | –reports }
options:
-h, –help
Show this help message and exit.
Mutually Exclusive Options:
–clusters
Displays the names of all clusters that InsightIQ is monitoring.
–breakouts
Displays the names of all breakouts that InsightIQ supports for file system analytic data modules for the specified cluster. Each data module supports a subset of breakouts.
–data-modules
Displays the names of all data modules that InsightIQ supports for file system analytic reports. Each data module supports a subset of breakouts.
–filters
Displays the names of all filters that have been created for the specified cluster.
–reports
Displays the names of all available file system analytic reports that have been created for the specified cluster.
Example output:
The following command displays the names of all file-system reports created for cluster1:iiq_data_export fsa list –reports cluster1
Available Reports for: cluster1 Time Zone: EST
ID FSA Job Start FSA Job End Size
—————————————————————————————————-
001 Sep 01 2015, 10:00 PM Sep 01 2015, 11:21 PM 150.12M
002 Sep 02 2015, 10:00 PM Sep 02 2015, 10:36 PM 152.12M
003 Sep 03 2015, 10:00 PM Sep 03 2015, 11:06 PM 151.88M
004 Sep 04 2015, 10:00 PM Sep 04 2015, 10:55 PM 154.12M
005 Sep 05 2015, 10:00 PM Sep 05 2015, 11:12 PM 153.00M
006 Sep 06 2015, 10:00 PM Sep 06 2015, 11:10 PM 152.50M
007 Sep 07 2015, 10:00 PM Sep 07 2015, 01:17 AM 156.12M
008 Sep 08 2015, 10:00 PM Sep 08 2015, 11:03 PM 152.00M
—————————————————————————————————-
iiq_data_export fsa export
iiq_data export fsa export usage/switches:
iiq_data_export fsa export [-h] -c -d -o [-b ] [-m ] [-n ] [-p ] [-r [: [: …]]] [-z ]
Required switches:
-c , –cluster
Exports data about the specified cluster.
-d , –data-module
Exports the specified data module.
The following module values are valid:
Module description | Module Value |
---|---|
File Count by Logical Size | file_count_by_logical_size |
File Count by Last Modified | file_count_by_modified_time |
File Count by Physical Size | file_count_by_physical_size |
Top Directories | top_directories |
Top Files | top_files |
-o , –report
Exports data contained in the specified report.
Option switches:
-h, –help
Show this help message and exit.
-b , –breakout-by
The following breakout values are valid:
Breakout description | Breakout Value |
---|---|
Accessed Time | atime |
Created Time | btime |
Modified Time | ctime |
Directory | directory |
Disk Pool | disk_pool |
File Extension | path_ext |
Logical Size | log_size |
Physical Size | phys_size |
User Attribute | attribute |
-m , –number-breakouts
Specifies the number of breakout components to include for each breakout. For example, if you broke out a data module by directory, specifying –number-breakouts 5 would export data about the top 5 directories. The default number is 12.
-n , –name
Specifies a name for the .csv file.
A time stamp is appended to the specified name. If no name is specified, the file is named according to the following pattern:
_[_][-]_.csv
-p , –path
Specifies a directory to store the .csv file.
-z , –comp-report
Instead of exporting the data contained in the report specified by the –report option, exports the comparison between the older report and the newer report specified by the –report and –comp-report options.
-r [: [: …]], –filter-rule [: [: …]]
Applies the specified filter rule to the exported data. To specify a filter, you must specify the breakout that the filter applies to
The following filter values are valid:
Breakout description | Breakout value | Filter values |
---|---|---|
Accessed Time | atime | -60 Specifies 0:00:00 – 0:01:00 -3600 Specifies 0:01:00 – 1:00:00 -86400 Specifies 1:00:00 – 1 day -604800 Specifies 1 day – 7 days -2592000 Specifies 7 days – 30 days -5184000 Specifies 30 days – 60 days -7776000 Specifies 60 days – 90 days -15552000 Specifies 90 days – 180 days -31536000 Specifies 180 days – 365 days -63072000 Specifies 365 days – 730 days -1328431946 Specifies 730 days – Infinite |
Example: –filter-rule atime:-3600 |
||
Modified Time | ctime | -60 Specifies 0:00:00 – 0:01:00 -3600 Specifies 0:01:00 – 1:00:00 -86400 Specifies 1:00:00 – 1 day -604800 Specifies 1 day – 7 days -2592000 Specifies 7 days – 30 days -5184000 Specifies 30 days – 60 days -7776000 Specifies 60 days – 90 days -15552000 Specifies 90 days – 180 days -31536000 Specifies 180 days – 365 days -63072000 Specifies 365 days – 730 days -1328431946 Specifies 730 days – Infinite |
Example: –filter-rule ctime:-3600 |
||
Directory | directory | The path of a directory starting with /ifs |
Example: –filter-rule directory:/ifs/dir1 |
||
Disk Pool | disk_pool | The name of a disk pool |
Example: –filter-rule disk_pool:pool1 |
||
File Extension | path_ext | The name of a file extension |
Example: –filter-rule path_ext:(pdf) |
||
Logical Size | log_size | A range of sizes in the following format: – 0 Specifies 0 byes – 8 KB 8192 Specifies 8 KB – 128 KB 131072 Specifies 128 KB – 1 MB 1048576 Specifies 1 MB – 10 MB 10485760 Specifies 10 MB – 100 MB 104857600 Specifies 100 MB – 1 GB 1073741824 Specifies 1 GB – 10 GB 10737418240 Specifies 10 GB – 100 GB 107374182400 Specifies 100 GB – 1 TB 1099511627776 Specifies 1 TB – Infinite |
Example: –filter-rule log_size:1GB-10GB |
||
Physical Size | phys_size | A range of sizes in the following format: – 0 Specifies 0 byes – 8 KB 8192 Specifies 8 KB – 128 KB 131072 Specifies 128 KB – 1 MB 1048576 Specifies 1 MB – 10 MB 10485760 Specifies 10 MB – 100 MB 104857600 Specifies 100 MB – 1 GB 1073741824 Specifies 1 GB – 10 GB 10737418240 Specifies 10 GB – 100 GB 107374182400 Specifies 100 GB – 1 TB 1099511627776 Specifies 1 TB – Infinite |
Example: –filter-rule phys_size: 1GB-10GB |
||
User Attribute | attribute | The name of a user attribute defined on the cluster. All characters are valid. |
Example of exporting InsightIQ file system analytics (FSA) modules to CSV:
Module: File Count query
#iiq_data_export fsa export –cluster Cluster1–data-module file_count_by_physical_size –report 002Output:
Successfully exported data to:
file_count_by_physical_size_Cluster1_002_1442429501.csv#cat file_count_by_physical_size_Cluster1_002_1442429501.csv
[no_filters],Totals,0 – 8192 bytes (file_count),8192 – 131072 bytes (file_count),131072 – 1048576 bytes (file_count),1048576 – 10485760 bytes (file_count),10485760 – 104857600 bytes (file_count),104857600 – 1073741824 bytes (file_count),1073741824 – 10737418240 bytes (file_count),10737418240 – 107374182400 bytes (file_count),107374182400 – 1099511627776 bytes (file_count),1099511627776 – Infinite bytes (file_count),report_date: 1442368827,ALL,4487319,649,56929,349275,3971841,34802,24204,27153,18833,3623,10
Output after formatting:
Totals | 4,487,319 |
---|---|
0 – 8192 bytes (file_count) | 649 |
8192 – 131072 bytes (file_count) | 56,929 |
131072 – 1048576 bytes (file_count) | 349,275 |
1048576 – 10485760 bytes (file_count) | 3,971,841 |
10485760 – 104857600 bytes (file_count) | 34,802 |
104857600 – 1073741824 bytes (file_count) | 24,204 |
1073741824 – 10737418240 bytes (file_count) | 27,153 |
10737418240 – 107374182400 bytes (file_count) | 18,833 |
107374182400 – 1099511627776 bytes (file_count) | 3,623 |
1099511627776 – Infinite bytes (file_count) | 10 |
Module: Top directories size
(by default it list 12 directories only unless you use –m then you can specify up to 1,000 directories)
# iiq_data_export fsa export –cluster Cluster1–data-module top_directories –report 002
Output:
Successfully exported data to:
top_dirs_btime_Cluster1_002_1442430835.csv
#cat top_dirs_btime_Cluster1_002_1442430835.csv
path:[no_filters],atime (unix),btime (unix),ctime (unix),report_date: 1442368827 Cluster1/dir01/dir01,1441783031,1415174737,1441783031
Cluster1/dir01/dir02,1441781797,1415174364,1441781797
Cluster1/dir01/dir03,1441778365,1415174022,1441778365
Cluster1/dir01/dir04,1441767600,1415173808,1441767600
Cluster1/dir01/dir05,1441778832,1415173760,1441778832
Cluster1/dir01/dir06,1441773114,1415173755,1441773114
Cluster1/dir01/dir07,1441782034,1415173381,1441782034
Cluster1/dir01/dir08,1441777596,1415173261,1441777596
Cluster1/dir01/dir09,1441770935,1415171817,1441770935
Cluster1/dir01/dir10,1441781832,1415170648,1441781832
Cluster1/dir01/dir11,1442366094,1415170066,1442366094
Cluster1/dir01/dir12,1441765803,1415169922,1441765803
Output after formatting:
Directory (Sorted descending by Create time) | access time (atime) | atime (converted to date) | atime (days) | created time (btime) | btime (converted to date) | btime (days) | modify time (ctime) | ctime (converted to date) | ctime (days) |
---|---|---|---|---|---|---|---|---|---|
Cluster1/dir01/dir01 | 1441783031 | 9/9/15 7:17 AM | 6.70 | 1415174737 | 11/5/14 8:05 AM | 314.66 | 1441783031 | 9/9/15 7:17 AM | 6.70 |
Cluster1/dir01/dir02 | 1441781797 | 9/9/15 6:56 AM | 6.71 | 1415174364 | 11/5/14 7:59 AM | 314.67 | 1441781797 | 9/9/15 6:56 AM | 6.71 |
Cluster1/dir01/dir03 | 1441778365 | 9/9/15 5:59 AM | 6.75 | 1415174022 | 11/5/14 7:53 AM | 314.67 | 1441778365 | 9/9/15 5:59 AM | 6.75 |
Cluster1/dir01/dir04 | 1441767600 | 9/9/15 3:00 AM | 6.88 | 1415173808 | 11/5/14 7:50 AM | 314.67 | 1441767600 | 9/9/15 3:00 AM | 6.88 |
Cluster1/dir01/dir05 | 1441778832 | 9/9/15 6:07 AM | 6.74 | 1415173760 | 11/5/14 7:49 AM | 314.67 | 1441778832 | 9/9/15 6:07 AM | 6.74 |
Cluster1/dir01/dir06 | 1441773114 | 9/9/15 4:31 AM | 6.81 | 1415173755 | 11/5/14 7:49 AM | 314.67 | 1441773114 | 9/9/15 4:31 AM | 6.81 |
Cluster1/dir01/dir07 | 1441782034 | 9/9/15 7:00 AM | 6.71 | 1415173381 | 11/5/14 7:43 AM | 314.68 | 1441782034 | 9/9/15 7:00 AM | 6.71 |
Cluster1/dir01/dir08 | 1441777596 | 9/9/15 5:46 AM | 6.76 | 1415173261 | 11/5/14 7:41 AM | 314.68 | 1441777596 | 9/9/15 5:46 AM | 6.76 |
Cluster1/dir01/dir09 | 1441770935 | 9/9/15 3:55 AM | 6.84 | 1415171817 | 11/5/14 7:16 AM | 314.70 | 1441770935 | 9/9/15 3:55 AM | 6.84 |
Cluster1/dir01/dir10 | 1441781832 | 9/9/15 6:57 AM | 6.71 | 1415170648 | 11/5/14 6:57 AM | 314.71 | 1441781832 | 9/9/15 6:57 AM | 6.71 |
Cluster1/dir01/dir11 | 1442366094 | 9/16/15 1:14 AM | -0.05 | 1415170066 | 11/5/14 6:47 AM | 314.72 | 1442366094 | 9/16/15 1:14 AM | -0.05 |
Cluster1/dir01/dir12 | 1441765803 | 9/9/15 2:30 AM | 6.90 | 1415169922 | 11/5/14 6:45 AM | 314.72 | 1441765803 | 9/9/15 2:30 AM | 6.90 |
Note:
* To convert unix time to data use the following formula (unixtime/(60*60*24)+”1/1/1970″
* to calculate the number of days we deducted the unix date from this report date which is 9/16/2015
Module: Top files size
(by default it list 12 files only unless you use –m then you can specify up to 1,000 files)
#iiq_data_export fsa export –cluster Cluster1–data-module top_files –report 002
Output:
Successfully exported data to:
top_files_btime_Cluster1_002_1442433280.csv
#cat top_files_btime_Cluster1_002_1442433280.csv
path:[no_filters],atime (unix),btime (unix),ctime (unix),log_size (bytes),phys_size (bytes),report_date: 1442368828 Cluster1/dir01/dir01/file01.pdf,1305942888,1421848937,1421862576,207378,780800
Cluster1/dir01/dir01/file02.pdf,1305939742,1421848937,1421862576,253591,821760
Cluster1/dir01/dir01/file03.pdf,1305941284,1421848936,1421862576,262503,838144
Cluster1/dir01/dir01/file04.pdf,1305904200,1421848936,1421862576,1241200,1812992
Cluster1/dir01/dir01/file5.pdf,1305906624,1421848936,1421862576,2680124,3779072
Cluster1/dir01/dir01/file06.pdf,1421113620,1421848935,1421862576,710,43520
Cluster1/dir01/dir01/file07.pdf,1305942306,1421848935,1421862576,207378,780800
Cluster1/dir01/dir01/file08.pdf,1417556492,1421848935,1421862576,770,43520
Cluster1/dir01/dir01/file09.pdf,1417556361,1421848935,1421862576,633,43520
Cluster1/dir01/dir01/file10.pdf,1305907374,1421848934,1421862576,207378,780800
Cluster1/dir01/dir01/file11.pdf,1305944776,1421848934,1421862577,2748940,3844608
Cluster1/dir01/dir01/file12.pdf,1305944776,1421848933,1421862577,262503,838144
Output after formatting:
Directory (Sorted descending by Create time) | access time (atime) | atime (converted to date) | atime (days) | created time (btime) | btime (converted to date) | btime (days) | modify time (ctime) | ctime (converted to date) | ctime (days) | log_size (bytes) | phys_size (bytes) |
---|---|---|---|---|---|---|---|---|---|---|---|
Cluster1/dir01/dir01/file01.pdf | 1305942888 | 5/21/11 1:54 AM | 1578.92 | 1421848937 | 1/21/15 2:02 PM | 237.42 | 1421862576 | 1/21/15 5:49 PM | 237.26 | 207,378 | 780,800 |
Cluster1/dir01/dir01/file02.pdf | 1305939742 | 5/21/11 1:02 AM | 1578.96 | 1421848937 | 1/21/15 2:02 PM | 237.42 | 1421862576 | 1/21/15 5:49 PM | 237.26 | 253,591 | 821,760 |
Cluster1/dir01/dir01/file03.pdf | 1305941284 | 5/21/11 1:28 AM | 1578.94 | 1421848936 | 1/21/15 2:02 PM | 237.42 | 1421862576 | 1/21/15 5:49 PM | 237.26 | 262,503 | 838,144 |
Cluster1/dir01/dir01/file04.pdf | 1305904200 | 5/20/11 3:10 PM | 1579.37 | 1421848936 | 1/21/15 2:02 PM | 237.42 | 1421862576 | 1/21/15 5:49 PM | 237.26 | 1,241,200 | 1,812,992 |
Cluster1/dir01/dir01/file05.pdf | 1305906624 | 5/20/11 3:50 PM | 1579.34 | 1421848936 | 1/21/15 2:02 PM | 237.42 | 1421862576 | 1/21/15 5:49 PM | 237.26 | 2,680,124 | 3,779,072 |
Cluster1/dir01/dir01/file06.pdf | 1421113620 | 1/13/15 1:47 AM | 245.93 | 1421848935 | 1/21/15 2:02 PM | 237.42 | 1421862576 | 1/21/15 5:49 PM | 237.26 | 710 | 43,520 |
Cluster1/dir01/dir01/file07.pdf | 1305942306 | 5/21/11 1:45 AM | 1578.93 | 1421848935 | 1/21/15 2:02 PM | 237.42 | 1421862576 | 1/21/15 5:49 PM | 237.26 | 207,378 | 780,800 |
Cluster1/dir01/dir01/file08.pdf | 1417556492 | 12/2/14 9:41 PM | 287.10 | 1421848935 | 1/21/15 2:02 PM | 237.42 | 1421862576 | 1/21/15 5:49 PM | 237.26 | 770 | 43,520 |
Cluster1/dir01/dir01/file09.pdf | 1417556361 | 12/2/14 9:39 PM | 287.10 | 1421848935 | 1/21/15 2:02 PM | 237.42 | 1421862576 | 1/21/15 5:49 PM | 237.26 | 633 | 43,520 |
Cluster1/dir01/dir01/file10.pdf | 1305907374 | 5/20/11 4:02 PM | 1579.33 | 1421848934 | 1/21/15 2:02 PM | 237.42 | 1421862576 | 1/21/15 5:49 PM | 237.26 | 207,378 | 780,800 |
Cluster1/dir01/dir01/file11.pdf | 1305944776 | 5/21/11 2:26 AM | 1578.90 | 1421848934 | 1/21/15 2:02 PM | 237.42 | 1421862577 | 1/21/15 5:49 PM | 237.26 | 2,748,940 | 3,844,608 |
Cluster1/dir01/dir01/file12.pdf | 1305944776 | 5/21/11 2:26 AM | 1578.90 | 1421848933 | 1/21/15 2:02 PM | 237.42 | 1421862577 | 1/21/15 5:49 PM | 237.26 | 262,503 | 838,144 |
Note:
* To convert unix time to data use the following formula (unixtime/(60*60*24)+”1/1/1970″
* to calculate the number of days we deducted the unix date from this report date which is 9/16/2015