つかわれてないSSDがあるのでL2ARCとしてつかってみた。
これで追加ok↓
zpool add tank cache /dev/ada4
で、とりあえずSMARTでも調べてみるかとsmartctl -ax /dev/ada4ってしたらOSごとプチフリーズした。
もどってきたけどzpool statusでみると REMOVED になってた。
% zpool status tank
pool: tank
state: ONLINE
scan: scrub repaired 0 in 7h6m with 0 errors on Wed Nov 30 10:52:13 2011
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
ada1s2 ONLINE 0 0 0
ada2s2 ONLINE 0 0 0
ada3s2 ONLINE 0 0 0
logs
label/zil ONLINE 0 0 0
cache
12789752023354324186 REMOVED 0 0 0 was /dev/ada4
errors: No known data errors
いったんL2ARCから外す。
zpool remove tank cache /dev/ada4
なんか返事がかえってこないし、CPUもブンブン回っている。
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
0 root -88 0 0K 5184K CPU3 3 1:25 100.00% [kernel{thread taskq}]
しばらくしたら正常になったが、このデバイスにはSMARTしちゃいけないみたいだ。朝、マシンのZFS関係がのきなみzfsのロックでひっかかって使用不能状態になってた。夜中のdailyでSMARTで値を監視しているところでハングアップしたようだ。
dailyでSMARTをとらないようにして一晩経過。なかなかL2ARCは効いているようだ。
zpool iostat -v tankの結果
capacity operations bandwidth
pool alloc free read write read write
----------- ----- ----- ----- ----- ----- -----
tank 885G 1.07T 12 18 176K 339K
raidz1 885G 1.07T 12 17 176K 311K
ada1s2 - - 4 8 135K 157K
ada2s2 - - 4 8 136K 157K
ada3s2 - - 4 8 135K 157K
logs - - - - - -
label/zil 1.19M 1.95G 0 0 13 28.0K
cache - - - - - -
ada4 21.1G 98.2G 9 4 69.1K 283K
----------- ----- ----- ----- ----- ----- -----
arc_summaryの結果
------------------------------------------------------------------------ ZFS Subsystem Report Fri Dec 9 10:35:44 2011 ------------------------------------------------------------------------ System Memory: 1.58% 172.32 MiB Active, 2.54% 276.75 MiB Inact 83.34% 8.86 GiB Wired, 0.03% 3.69 MiB Cache 12.49% 1.33 GiB Free, 0.01% 1.44 MiB Gap Real Installed: 12.00 GiB Real Available: 91.54% 10.98 GiB Real Managed: 96.82% 10.64 GiB Logical Total: 12.00 GiB Logical Used: 86.65% 10.40 GiB Logical Free: 13.35% 1.60 GiB Kernel Memory: 280.84 MiB DATA: 90.71% 254.76 MiB TEXT: 9.29% 26.08 MiB KMAP: 5.63 GiB FREE: 21.38% 1.20 GiB Page: 1 ------------------------------------------------------------------------ ARC Summary: (THROTTLED) Storage pool Version: 28 Filesystem Version: 5 Memory Throttle Count: 1 ARC Misc: Deleted: 741.62k Recycle Misses: 546.38k Mutex Misses: 2.11k Evict Skips: 2.11k ARC Size: 54.46% 3.27 GiB Target Size: (Adaptive) 84.34% 5.06 GiB Min Size (Hard Limit): 66.67% 4.00 GiB Max Size (High Water): 1:1 6.00 GiB ARC Size Breakdown: Recently Used Cache Size: 46.35% 2.35 GiB Frequently Used Cache Size: 53.65% 2.71 GiB ARC Hash Breakdown: Elements Max: 810.84k Elements Current: 100.00% 810.84k Collisions: 1.23m Chain Max: 13 Chains: 213.53k Page: 2 ------------------------------------------------------------------------ ARC Efficiency: 27.16m Cache Hit Ratio: 92.91% 25.23m Cache Miss Ratio: 7.09% 1.93m Actual Hit Ratio: 84.06% 22.83m Data Demand Efficiency: 92.59% 10.07m Data Prefetch Efficiency: 29.31% 107.15k CACHE HITS BY CACHE LIST: Anonymously Used: 5.53% 1.39m Most Recently Used: 13.31% 3.36m Most Frequently Used: 77.17% 19.47m Most Recently Used Ghost: 1.73% 437.57k Most Frequently Used Ghost: 2.27% 571.79k CACHE HITS BY DATA TYPE: Demand Data: 36.96% 9.33m Prefetch Data: 0.12% 31.40k Demand Metadata: 53.51% 13.50m Prefetch Metadata: 9.41% 2.37m CACHE MISSES BY DATA TYPE: Demand Data: 38.77% 746.85k Prefetch Data: 3.93% 75.75k Demand Metadata: 43.50% 837.98k Prefetch Metadata: 13.79% 265.66k Page: 3 ------------------------------------------------------------------------ L2 ARC Summary: (HEALTHY) Passed Headroom: 1.69m Tried Lock Failures: 2.13m IO In Progress: 2.29k Low Memory Aborts: 13 Free on Write: 2.37k Writes While Full: 1.62k R/W Clashes: 320 Bad Checksums: 0 IO Errors: 0 SPA Mismatch: 472.43m L2 ARC Size: (Adaptive) 18.11 GiB Header Size: 0.43% 79.62 MiB L2 ARC Breakdown: 1.93m Hit Ratio: 36.49% 702.93k Miss Ratio: 63.51% 1.22m Feeds: 78.65k L2 ARC Buffer: Bytes Scanned: 188.16 TiB Buffer Iterations: 78.65k List Iterations: 5.01m NULL List Iterations: 614.57k L2 ARC Writes: Writes Sent: 100.00% 12.04k Page: 4 ------------------------------------------------------------------------ File-Level Prefetch: (HEALTHY) DMU Efficiency: 90.34m Hit Ratio: 74.24% 67.07m Miss Ratio: 25.76% 23.27m Colinear: 23.27m Hit Ratio: 0.01% 2.54k Miss Ratio: 99.99% 23.27m Stride: 66.75m Hit Ratio: 99.94% 66.70m Miss Ratio: 0.06% 41.48k DMU Misc: Reclaim: 23.27m Successes: 0.36% 83.31k Failures: 99.64% 23.18m Streams: 364.79k +Resets: 0.08% 300 -Resets: 99.92% 364.49k Bogus: 0 Page: 5 ------------------------------------------------------------------------ VDEV Cache Summary: 1.00m Hit Ratio: 60.34% 605.51k Miss Ratio: 29.96% 300.61k Delegations: 9.70% 97.35k Page: 6 ------------------------------------------------------------------------ ZFS Tunable (sysctl): kern.maxusers=384 vfs.zfs.l2c_only_size=13392641536 vfs.zfs.mfu_ghost_data_lsize=670471168 vfs.zfs.mfu_ghost_metadata_lsize=765786112 vfs.zfs.mfu_ghost_size=1436257280 vfs.zfs.mfu_data_lsize=848583168 vfs.zfs.mfu_metadata_lsize=379664896 vfs.zfs.mfu_size=1382747136 vfs.zfs.mru_ghost_data_lsize=2289632256 vfs.zfs.mru_ghost_metadata_lsize=1683590656 vfs.zfs.mru_ghost_size=3973222912 vfs.zfs.mru_data_lsize=1204788736 vfs.zfs.mru_metadata_lsize=153126912 vfs.zfs.mru_size=1384736256 vfs.zfs.anon_data_lsize=0 vfs.zfs.anon_metadata_lsize=0 vfs.zfs.anon_size=2969600 vfs.zfs.l2arc_norw=1 vfs.zfs.l2arc_feed_again=1 vfs.zfs.l2arc_noprefetch=1 vfs.zfs.l2arc_feed_min_ms=200 vfs.zfs.l2arc_feed_secs=1 vfs.zfs.l2arc_headroom=10 vfs.zfs.l2arc_write_boost=8388608 vfs.zfs.l2arc_write_max=8388608 vfs.zfs.arc_meta_limit=1610612736 vfs.zfs.arc_meta_used=1455369960 vfs.zfs.arc_min=4294967296 vfs.zfs.arc_max=6442450944 vfs.zfs.dedup.prefetch=1 vfs.zfs.mdcomp_disable=0 vfs.zfs.write_limit_override=0 vfs.zfs.write_limit_inflated=35384819712 vfs.zfs.write_limit_max=1474367488 vfs.zfs.write_limit_min=33554432 vfs.zfs.write_limit_shift=3 vfs.zfs.no_write_throttle=0 vfs.zfs.zfetch.array_rd_sz=1048576 vfs.zfs.zfetch.block_cap=256 vfs.zfs.zfetch.min_sec_reap=2 vfs.zfs.zfetch.max_streams=8 vfs.zfs.prefetch_disable=0 vfs.zfs.mg_alloc_failures=8 vfs.zfs.check_hostid=1 vfs.zfs.recover=0 vfs.zfs.txg.synctime_ms=1000 vfs.zfs.txg.timeout=5 vfs.zfs.scrub_limit=10 vfs.zfs.vdev.cache.bshift=16 vfs.zfs.vdev.cache.size=104857600 vfs.zfs.vdev.cache.max=16384 vfs.zfs.vdev.write_gap_limit=4096 vfs.zfs.vdev.read_gap_limit=32768 vfs.zfs.vdev.aggregation_limit=131072 vfs.zfs.vdev.ramp_rate=2 vfs.zfs.vdev.time_shift=6 vfs.zfs.vdev.min_pending=4 vfs.zfs.vdev.max_pending=10 vfs.zfs.vdev.bio_flush_disable=0 vfs.zfs.cache_flush_disable=0 vfs.zfs.zil_replay_disable=0 vfs.zfs.zio.use_uma=1 vfs.zfs.version.zpl=5 vfs.zfs.version.spa=28 vfs.zfs.version.acl=1 vfs.zfs.debug=0 vfs.zfs.super_owner=0 vm.kmem_size=8589934592 vm.kmem_size_scale=1 vm.kmem_size_min=0 vm.kmem_size_max=329853485875 Page: 7 ------------------------------------------------------------------------
