つかわれてないSSDがあるのでL2ARCとしてつかってみた。

これで追加ok↓

zpool add tank cache /dev/ada4

で、とりあえずSMARTでも調べてみるかとsmartctl -ax /dev/ada4ってしたらOSごとプチフリーズした。

もどってきたけどzpool statusでみると REMOVED になってた。

% zpool status  tank
  pool: tank
 state: ONLINE
  scan: scrub repaired 0 in 7h6m with 0 errors on Wed Nov 30 10:52:13 2011
config:

        NAME                    STATE     READ WRITE CKSUM
        tank                    ONLINE       0     0     0
          raidz1-0              ONLINE       0     0     0
            ada1s2              ONLINE       0     0     0
            ada2s2              ONLINE       0     0     0
            ada3s2              ONLINE       0     0     0
        logs
          label/zil             ONLINE       0     0     0
        cache
          12789752023354324186  REMOVED      0     0     0  was /dev/ada4

errors: No known data errors

いったんL2ARCから外す。

zpool remove tank cache /dev/ada4

なんか返事がかえってこないし、CPUもブンブン回っている。


  PID USERNAME      PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
    0 root          -88    0     0K  5184K CPU3    3   1:25 100.00% [kernel{thread taskq}]

しばらくしたら正常になったが、このデバイスにはSMARTしちゃいけないみたいだ。朝、マシンのZFS関係がのきなみzfsのロックでひっかかって使用不能状態になってた。夜中のdailyでSMARTで値を監視しているところでハングアップしたようだ。

dailyでSMARTをとらないようにして一晩経過。なかなかL2ARCは効いているようだ。

zpool iostat -v tankの結果

                capacity     operations    bandwidth
pool         alloc   free   read  write   read  write
-----------  -----  -----  -----  -----  -----  -----
tank          885G  1.07T     12     18   176K   339K
  raidz1      885G  1.07T     12     17   176K   311K
    ada1s2       -      -      4      8   135K   157K
    ada2s2       -      -      4      8   136K   157K
    ada3s2       -      -      4      8   135K   157K
logs             -      -      -      -      -      -
  label/zil  1.19M  1.95G      0      0     13  28.0K
cache            -      -      -      -      -      -
  ada4       21.1G  98.2G      9      4  69.1K   283K
-----------  -----  -----  -----  -----  -----  -----

arc_summaryの結果

------------------------------------------------------------------------
ZFS Subsystem Report				Fri Dec  9 10:35:44 2011
------------------------------------------------------------------------

System Memory:
	1.58%	172.32	MiB Active,	2.54%	276.75	MiB Inact
	83.34%	8.86	GiB Wired,	0.03%	3.69	MiB Cache
	12.49%	1.33	GiB Free,	0.01%	1.44	MiB Gap

	Real Installed:				12.00	GiB
	Real Available:			91.54%	10.98	GiB
	Real Managed:			96.82%	10.64	GiB

	Logical Total:				12.00	GiB
	Logical Used:			86.65%	10.40	GiB
	Logical Free:			13.35%	1.60	GiB

Kernel Memory:					280.84	MiB
	DATA:				90.71%	254.76	MiB
	TEXT:				9.29%	26.08	MiB

	KMAP:					5.63	GiB
	FREE:				21.38%	1.20	GiB
								Page:  1
------------------------------------------------------------------------

ARC Summary: (THROTTLED)
	Storage pool Version:			28
	Filesystem Version:			5
	Memory Throttle Count:			1

ARC Misc:
	Deleted:				741.62k
	Recycle Misses:				546.38k
	Mutex Misses:				2.11k
	Evict Skips:				2.11k

ARC Size:				54.46%	3.27	GiB
	Target Size: (Adaptive)		84.34%	5.06	GiB
	Min Size (Hard Limit):		66.67%	4.00	GiB
	Max Size (High Water):		1:1	6.00	GiB

ARC Size Breakdown:
	Recently Used Cache Size:	46.35%	2.35	GiB
	Frequently Used Cache Size:	53.65%	2.71	GiB

ARC Hash Breakdown:
	Elements Max:				810.84k
	Elements Current:		100.00%	810.84k
	Collisions:				1.23m
	Chain Max:				13
	Chains:					213.53k
								Page:  2
------------------------------------------------------------------------

ARC Efficiency:					27.16m
	Cache Hit Ratio:		92.91%	25.23m
	Cache Miss Ratio:		7.09%	1.93m
	Actual Hit Ratio:		84.06%	22.83m

	Data Demand Efficiency:		92.59%	10.07m
	Data Prefetch Efficiency:	29.31%	107.15k

	CACHE HITS BY CACHE LIST:
	  Anonymously Used:		5.53%	1.39m
	  Most Recently Used:		13.31%	3.36m
	  Most Frequently Used:		77.17%	19.47m
	  Most Recently Used Ghost:	1.73%	437.57k
	  Most Frequently Used Ghost:	2.27%	571.79k

	CACHE HITS BY DATA TYPE:
	  Demand Data:			36.96%	9.33m
	  Prefetch Data:		0.12%	31.40k
	  Demand Metadata:		53.51%	13.50m
	  Prefetch Metadata:		9.41%	2.37m

	CACHE MISSES BY DATA TYPE:
	  Demand Data:			38.77%	746.85k
	  Prefetch Data:		3.93%	75.75k
	  Demand Metadata:		43.50%	837.98k
	  Prefetch Metadata:		13.79%	265.66k
								Page:  3
------------------------------------------------------------------------

L2 ARC Summary: (HEALTHY)
	Passed Headroom:			1.69m
	Tried Lock Failures:			2.13m
	IO In Progress:				2.29k
	Low Memory Aborts:			13
	Free on Write:				2.37k
	Writes While Full:			1.62k
	R/W Clashes:				320
	Bad Checksums:				0
	IO Errors:				0
	SPA Mismatch:				472.43m

L2 ARC Size: (Adaptive)				18.11	GiB
	Header Size:			0.43%	79.62	MiB

L2 ARC Breakdown:				1.93m
	Hit Ratio:			36.49%	702.93k
	Miss Ratio:			63.51%	1.22m
	Feeds:					78.65k

L2 ARC Buffer:
	Bytes Scanned:				188.16	TiB
	Buffer Iterations:			78.65k
	List Iterations:			5.01m
	NULL List Iterations:			614.57k

L2 ARC Writes:
	Writes Sent:			100.00%	12.04k
								Page:  4
------------------------------------------------------------------------

File-Level Prefetch: (HEALTHY)

DMU Efficiency:					90.34m
	Hit Ratio:			74.24%	67.07m
	Miss Ratio:			25.76%	23.27m

	Colinear:				23.27m
	  Hit Ratio:			0.01%	2.54k
	  Miss Ratio:			99.99%	23.27m

	Stride:					66.75m
	  Hit Ratio:			99.94%	66.70m
	  Miss Ratio:			0.06%	41.48k

DMU Misc:
	Reclaim:				23.27m
	  Successes:			0.36%	83.31k
	  Failures:			99.64%	23.18m

	Streams:				364.79k
	  +Resets:			0.08%	300
	  -Resets:			99.92%	364.49k
	  Bogus:				0
								Page:  5
------------------------------------------------------------------------

VDEV Cache Summary:				1.00m
	Hit Ratio:			60.34%	605.51k
	Miss Ratio:			29.96%	300.61k
	Delegations:			9.70%	97.35k
								Page:  6
------------------------------------------------------------------------

ZFS Tunable (sysctl):
	kern.maxusers=384
	vfs.zfs.l2c_only_size=13392641536
	vfs.zfs.mfu_ghost_data_lsize=670471168
	vfs.zfs.mfu_ghost_metadata_lsize=765786112
	vfs.zfs.mfu_ghost_size=1436257280
	vfs.zfs.mfu_data_lsize=848583168
	vfs.zfs.mfu_metadata_lsize=379664896
	vfs.zfs.mfu_size=1382747136
	vfs.zfs.mru_ghost_data_lsize=2289632256
	vfs.zfs.mru_ghost_metadata_lsize=1683590656
	vfs.zfs.mru_ghost_size=3973222912
	vfs.zfs.mru_data_lsize=1204788736
	vfs.zfs.mru_metadata_lsize=153126912
	vfs.zfs.mru_size=1384736256
	vfs.zfs.anon_data_lsize=0
	vfs.zfs.anon_metadata_lsize=0
	vfs.zfs.anon_size=2969600
	vfs.zfs.l2arc_norw=1
	vfs.zfs.l2arc_feed_again=1
	vfs.zfs.l2arc_noprefetch=1
	vfs.zfs.l2arc_feed_min_ms=200
	vfs.zfs.l2arc_feed_secs=1
	vfs.zfs.l2arc_headroom=10
	vfs.zfs.l2arc_write_boost=8388608
	vfs.zfs.l2arc_write_max=8388608
	vfs.zfs.arc_meta_limit=1610612736
	vfs.zfs.arc_meta_used=1455369960
	vfs.zfs.arc_min=4294967296
	vfs.zfs.arc_max=6442450944
	vfs.zfs.dedup.prefetch=1
	vfs.zfs.mdcomp_disable=0
	vfs.zfs.write_limit_override=0
	vfs.zfs.write_limit_inflated=35384819712
	vfs.zfs.write_limit_max=1474367488
	vfs.zfs.write_limit_min=33554432
	vfs.zfs.write_limit_shift=3
	vfs.zfs.no_write_throttle=0
	vfs.zfs.zfetch.array_rd_sz=1048576
	vfs.zfs.zfetch.block_cap=256
	vfs.zfs.zfetch.min_sec_reap=2
	vfs.zfs.zfetch.max_streams=8
	vfs.zfs.prefetch_disable=0
	vfs.zfs.mg_alloc_failures=8
	vfs.zfs.check_hostid=1
	vfs.zfs.recover=0
	vfs.zfs.txg.synctime_ms=1000
	vfs.zfs.txg.timeout=5
	vfs.zfs.scrub_limit=10
	vfs.zfs.vdev.cache.bshift=16
	vfs.zfs.vdev.cache.size=104857600
	vfs.zfs.vdev.cache.max=16384
	vfs.zfs.vdev.write_gap_limit=4096
	vfs.zfs.vdev.read_gap_limit=32768
	vfs.zfs.vdev.aggregation_limit=131072
	vfs.zfs.vdev.ramp_rate=2
	vfs.zfs.vdev.time_shift=6
	vfs.zfs.vdev.min_pending=4
	vfs.zfs.vdev.max_pending=10
	vfs.zfs.vdev.bio_flush_disable=0
	vfs.zfs.cache_flush_disable=0
	vfs.zfs.zil_replay_disable=0
	vfs.zfs.zio.use_uma=1
	vfs.zfs.version.zpl=5
	vfs.zfs.version.spa=28
	vfs.zfs.version.acl=1
	vfs.zfs.debug=0
	vfs.zfs.super_owner=0
	vm.kmem_size=8589934592
	vm.kmem_size_scale=1
	vm.kmem_size_min=0
	vm.kmem_size_max=329853485875
								Page:  7
------------------------------------------------------------------------

koie