ç»åãœãŒã¹ïŒ www.nikonsmallworld.com
çäœé²æ¢ã¯å°éã®æ€çŽ¢ãšã³ãžã³ã§ããã§ã«ä»¥åã«æžãããŠããŸãã ãŸããã©ã®æ€çŽ¢ãšã³ãžã³ã§ããã©ã®ããã«å転ãããŠãè¿ éã«åäœããã«ã¯ãæ€çŽ¢é åã®ãã¹ãŠã®æ©èœãèæ ®ããç¬èªã®ã€ã³ããã¯ã¹ãå¿ èŠã§ãã Habréã«é¢ããæåã®èšäºã§ã¯ãæ€çŽ¢ã€ã³ããã¯ã¹ã®çŸåšã®å®è£ ããã®éçºã®æŽå²ãããã³1ã€ãŸãã¯å¥ã®ãœãªã¥ãŒã·ã§ã³ãéžæããçç±ã«ã€ããŠèª¬æããŸãã å¹æçãª.NETã¢ã«ãŽãªãºã ã¯ç¥è©±ã§ã¯ãªããå°é£ã§çç£çãªçŸå®ã§ãã ããã·ã¥ããããåäœã®å§çž®ããã«ãã¬ãã«ã®åªå 床ãã£ãã·ã¥ã®äžçã«çªå ¥ããŸãã OïŒ1ïŒãããé«éãªæ€çŽ¢ãå¿ èŠãªå Žåã¯ã©ããªããŸããïŒ
ä»ã®èª°ãããã®åçã®åž¯ç¶ç±ç¹ã®äœçœ®ãç¥ããªãå Žåããããã...
垯ç¶ç±ç¹ãã€ã³ããã¯ã¹ããã³ããããæ¢ãçç±
ã·ã³ã°ã«ã¯ããµã€ãºãæ°ã¯ãŒãã®ããã¹ãã§ãã 垯ç¶ç±ç¹ã¯äºãã«éãªãåã£ãŠãããããååïŒè±èªã垯ç¶ç±ç¹-é±ãã¿ã€ã«ïŒã ãããã®ç¹å®ã®ãµã€ãºã¯å ¬éç§å¯ã§ã-4ã¯ãŒãã ãããšã5ïŒ ãŸããããã¯äŸåããŸãã ãã ãããã®å€ã§ãã»ãšãã©æå³ããªããã¹ãããã¯ãŒãã®æ§æãã¯ãŒããæ£èŠåããã¢ã«ãŽãªãºã ãããã³ãã®èšäºã®ãã¬ãŒã ã¯ãŒã¯ã§ã¯éèŠã§ã¯ãªããã®ä»ã®è©³çŽ°ã«äŸåããŸãã æåŸã«ããã®ã·ã³ã°ã«ã«åºã¥ããŠ64ãããããã·ã¥ãèšç®ããŸãããããå°æ¥ã·ã³ã°ã«ãšåŒã³ãŸãã
ããã¥ã¡ã³ãã®ããã¹ãã«ãããšãå€ãã®åž¯ç¶ç±ç¹ãäœæã§ããŸãã垯ç¶ç±ç¹ã®æ°ã¯ãããã¥ã¡ã³ãå ã®åèªã®æ°ã«å¹æµããŸãã
ããã¹ãïŒæååâ垯ç¶ç±ç¹ïŒuint64 []
è€æ°ã®åž¯ç¶ç±ç¹ãäžèŽããå Žåã2ã€ã®ããã¥ã¡ã³ãã¯ããã¥ã¡ã³ãã亀差ãããšæ³å®ããŸãã äžèŽãã垯ç¶ç±ç¹ã®æ°ãå€ãã»ã©ããã®ããã¥ã¡ã³ããã¢å ã®ããã¹ãã¯åãã«ãªããŸãã ã€ã³ããã¯ã¹ã¯ããã§ãã¯å¯Ÿè±¡ã®ããã¥ã¡ã³ããšã®å ±ééšåãæãå€ãããã¥ã¡ã³ããæ€çŽ¢ããŸãã
ç»åãœãŒã¹ïŒ ãŠã£ãããã£ã¢
垯ç¶ç±ç¹ã€ã³ããã¯ã¹ã䜿çšãããšã2ã€ã®äž»èŠãªæäœãå®è¡ã§ããŸãã
ããã¥ã¡ã³ãã®åž¯ç¶ç±ç¹ã«èå¥åãä»ããŠã€ã³ããã¯ã¹ãäœæããŸãã
index.AddïŒdocIdã垯ç¶ç±ç¹ïŒ
éè€ããããã¥ã¡ã³ãã®èå¥åã®ã©ã³ã¯ä»ããªã¹ããæ€çŽ¢ããŠè¡šç€ºããŸãã
index.SearchïŒã·ã³ã°ã«ïŒâïŒdocIdãã¹ã³ã¢ïŒ[]
ã©ã³ãã³ã°ã¢ã«ãŽãªãºã ã¯ãäžè¬ã«å¥ã®èšäºãšããŠäŸ¡å€ããããšæãã®ã§ãããã§ã¯èª¬æããŸããã
垯ç¶ç±ç¹ã®ã€ã³ããã¯ã¹ã¯ãSphinxãElasticããŸãã¯ãã以äžã®GoogleãYandexãªã©ã®ããç¥ãããŠãããã«ããã¹ãã®ã€ã³ããã¯ã¹ãšã¯å€§ããç°ãªããŸãã ãã¹ãŠã®ããã¹ãåŠçãåãé€ãããããã»ã¹ãããã³ããã¹ãå ã®åž¯ç¶ç±ç¹ã®ã·ãŒã±ã³ã¹ã«ã¯åœ±é¿ããŸããã äžæ¹ãæ€çŽ¢ã¯ãšãªã¯ãåèªãŸãã¯è€æ°ã®åèªã®ãã¬ãŒãºã§ã¯ãªããæ倧æ°åäžåã®ããã·ã¥ã§ããããã¹ãŠãåå¥ã«ã§ã¯ãªããå šäœãšããŠéèŠã§ãã
仮説çã«ã¯ã垯ç¶ç±ç¹ã€ã³ããã¯ã¹ã®ä»£ããã«ãã«ããã¹ãã€ã³ããã¯ã¹ã䜿çšã§ããŸãããéãã¯å€§ããããŸãã ããã€ãã®ããç¥ãããŠããããŒå€ã¹ãã¬ãŒãžã䜿çšããæãç°¡åãªæ¹æ³ã¯ã以äžã§èª¬æããŸãã ç§ãã¡ã¯ãShingleIndexãšåŒã°ããèªè»¢è»ã®å®è£
ãèŠãŠããŸãã
ãªãããããã®ã§ããïŒ ãããããªãã
- ããªã¥ãŒã ïŒ
- ããããã®ããã¥ã¡ã³ãããããŸãã çŸåšãçŽ6å5,000äžå°ããããŸãããä»å¹Žã¯æããã«ãããããå€ããªããŸãã
- ãŠããŒã¯ãªåž¯ç¶ç±ç¹ã®æ°ã¯é£èºçã«å¢å ããŠããããã§ã«æ°ååã«éããŠããŸãã ç§ãã¡ã¯1å ãåŸ ã£ãŠããŸãã
- ã¹ããŒã
- æ¥äžãå€ã®ã»ãã·ã§ã³äžã«ã30äžä»¶ä»¥äžã®ææžãåãã©ã®ã¢ãªãºã ã·ã¹ãã ãéããŠãã§ãã¯ãããŸãã ããã¯äººæ°ã®ããæ€çŽ¢ãšã³ãžã³ã®æšæºã§ã¯å°ãã§ããã調åã¯å€ãããŸããã
- ããã¥ã¡ã³ãã®äžææ§ã®æ€èšŒãæåãããã«ã¯ãã€ã³ããã¯ã¹ä»ãããã¥ã¡ã³ãã®æ°ããã§ãã¯å¯Ÿè±¡ã®ããã¥ã¡ã³ãã®æ°ããã倧ããããå¿ èŠããããŸãã ã€ã³ããã¯ã¹ã®çŸåšã®ããŒãžã§ã³ã¯ãå¹³åããŠ1ç§ããã4000ãè¶ ããäžçšåºŠã®ããã¥ã¡ã³ãã®é床ã§ãã£ã±ãã«ãªããŸãã
ãããŠãããã¯ãã¹ãŠ1å°ã®ãã·ã³äžã«ãããŸãïŒ ã¯ãã è€è£œã§ããŸããã¯ã©ã¹ã¿ãŒã®åçã·ã£ãŒãã£ã³ã°ã«åŸã ã«è¿ã¥ããŠããŸããã2005幎ããä»æ¥ãŸã§ã1å°ã®ãã·ã³ã®ã€ã³ããã¯ã¹ã¯æ éã«æ³šæããŠäžèšã®ãã¹ãŠã®åé¡ã«å¯ŸåŠã§ããŸããã
å¥åŠãªçµéš
ããããç§ãã¡ã¯ä»ãšãŠãçµéšãç©ãã§ããŸãã 奜ããšå¥œãŸãããšã«ããããããç§ãã¡ãæé·ããæé·ã®éçšã§ããŸããŸãªããšãè©ŠããŠããŸããã
ç»åãœãŒã¹ïŒ ãŠã£ãããã£ã¢
ãŸã第äžã«ãçµéšã®æµ ãèªè ã¯SQLããŒã¿ããŒã¹ã䜿çšããããšæãã§ãããã ããèããã®ã¯ããªãã ãã§ã¯ãããŸãããSQLã®å®è£ ã¯ãæ°å¹Žã«ããã£ãŠéåžžã«å°ããªã³ã¬ã¯ã·ã§ã³ãå®è£ ããã®ã«åœ¹ç«ã£ãŠããŸããã ããã«ãããããããããã«äœçŸäžãã®ããã¥ã¡ã³ãã«çŠç¹ãåœãŠãããã®ã§ãããã«å ãžé²ãå¿ èŠããããŸããã
ãåç¥ã®ããã«ã誰ãèªè»¢è»ãæããŠããããLevelDBã¯ãŸã å ¬éãããŠããªãããã2010幎ã«BerkeleyDBã«æ³šç®ããŸããã ãã¹ãŠãã¯ãŒã«ã§ã-é©åãªbtreeããã³ããã·ã¥ã¢ã¯ã»ã¹ã¡ãœãããšé·ãå±¥æŽãæã€æ°žç¶çãªçµã¿èŸŒã¿ããŒå€ããŒã¹ã 圌女ãšã®ãã¹ãŠã¯çŽ æŽããããã®ã§ãããã
- ããã·ã¥å®è£ ã®å Žåã2GBã®ããªã¥ãŒã ã«éãããšãåçŽã«æžå°ããŸããã ã¯ãããŸã 32ãããã¢ãŒãã§äœæ¥ããŠããŸããã
- B +ããªãŒã®å®è£ ã¯å®å®ããŠæ©èœããŸããããæ°ã®ã¬ãã€ããè¶ ããããªã¥ãŒã ã§ã¯ãæ€çŽ¢é床ãå€§å¹ ã«äœäžãå§ããŸããã
ã¿ã¹ã¯ã«åãããŠèª¿æŽããæ¹æ³ãèŠã€ããããªãã£ãããšãèªããªããã°ãªããŸããã ãã¶ãåé¡ã¯.net-bindingsã«ãããããã¯ãŸã çµãããªããã°ãªããŸããã§ããã æçµçã«ãBDBå®è£ ã¯ãã¡ã€ã³ã€ã³ããã¯ã¹ãåããåã«ãäžéã€ã³ããã¯ã¹ãšããŠSQLã®ä»£ãããšããŠäœ¿çšãããŸããã
æéãçµã¡ãŸããã 2014幎ã圌ãã¯LMDBãšLevelDBãè©ŠããŸãããããããå®è£ ããŸããã§ããã ç§ãã¡ã®åå¹³å䞻矩ç 究éšéã®ã¡ã³ããŒã¯ãRocksDBãã€ã³ããã¯ã¹ãšããŠäœ¿çšããŸããã äžèŠãçºèŠã§ããã ããããå°éã®è£å ã§ãã£ãŠãé ãè£å ãšå¹³å¡ãªæ€çŽ¢é床ã¯ããã¹ãŠãç¡ã«åž°ããŸããã
ç¬èªã®ã«ã¹ã¿ã ã€ã³ããã¯ã¹ãéçºããªãããäžèšã®ãã¹ãŠãè¡ããŸããã ãã®çµæã圌ã¯åé¡ã®è§£æ±ºã«éåžžã«åªãããã®ã«ãªã£ãããã以åã®ããã©ã°ããæŸæ£ãããããæ¹åããããšã«éäžããŸããã
ã€ã³ããã¯ã¹ã¬ã€ã€ãŒ
æåŸã«ãç§ãã¡ã¯ä»äœãæã£ãŠããŸããïŒ å®éã垯ç¶ç±ç¹ã®ã€ã³ããã¯ã¹ã¯ãäžå®ã®é·ãïŒ0ã128ãããïŒã®èŠçŽ ãæã€è€æ°ã®å±€ïŒé åïŒã§æ§æãããŸããããã¯ãå±€ã ãã§ãªããå¿ ããã8ã®åæ°ã§ã¯ãããŸããã
åã¬ã€ã€ãŒã圹å²ãæãããŸãã æ€çŽ¢ãé«éåãããã®ãã¹ããŒã¹ãç¯çŽãããã®ã䜿çšãããªããæ¬åœã«å¿ èŠãªãã®ããããŸãã æ€çŽ¢ã§ã®å šäœçãªå¹æãé«ããããã«ããããã説æããããšããŸãã
ç»åãœãŒã¹ïŒ ãŠã£ãããã£ã¢
1.ã€ã³ããã¯ã¹é å
äžè¬æ§ã倱ãããšãªããåäžã®ã·ã³ã°ã«ãããã¥ã¡ã³ãã«å²ãåœãŠãããŠãããšèããŸãã
ïŒdocIdâã·ã³ã°ã«ïŒ
ãã¢ã®èŠçŽ ãå Žæããšã«å ¥ãæ¿ããŸãïŒã€ã³ããã¯ã¹ã¯å®éã«ã¯ãå転ãããŠãããããå転ããŸãïŒã
ïŒã·ã³ã°ã«âdocIdïŒ
垯ç¶ç±ç¹ã®å€ã§ãœãŒãããã¬ã€ã€ãŒã圢æããŸãã ãªããªã ã·ã³ã°ã«ãšããã¥ã¡ã³ãèå¥åã®ãµã€ãºã¯äžå®ã§ããããã ãã€ããªæ€çŽ¢ãç解ããŠãã人ã¯èª°ã§ããã¡ã€ã«ã®OïŒlognïŒèªã¿åãå€ã®ãã¢ãèŠã€ããããšãã§ããŸãã æ¬åœã«ãããã ããããããã¯OïŒnïŒãããåªããŠããŸãã
ããã¥ã¡ã³ãã«è€æ°ã®åž¯ç¶ç±ç¹ãããå Žåãããã¥ã¡ã³ããããã®ãããªãã¢ãããã€ããããŸãã åãéçãæã€è€æ°ã®ããã¥ã¡ã³ããããå ŽåããããããŸãå€åããŸãããåãéçãæã€è€æ°ã®ãã¢ãé£ç¶ããŠååšããŸãã ã©ã¡ãã®å Žåããæ€çŽ¢ã¯åçã®æéã§è¡ãããŸãã
2.ã°ã«ãŒãã®é å
åã®ã¹ãããã®ã€ã³ããã¯ã¹ã®èŠçŽ ãã䟿å©ãªæ¹æ³ã§æ
éã«ã°ã«ãŒãã«åããŸãã ããšãã°ãã¯ã©ã¹ã¿ãŒã»ã¯ã¿ãŒã«åãŸãããã«ããããæ°ããã®ä»ã®ããªãã¯ãèæ
®ããã¢ãã±ãŒã·ã§ã³ãŠãããããã㯠ïŒèªã¿åãã4096ãã€ãïŒãå¹æçãªèŸæžã圢æããŸãã ãã®ãããªã°ã«ãŒãã®äœçœ®ã®åçŽãªé
åãååŸããŸãã
group_mapïŒããã·ã¥ïŒã·ã³ã°ã«ïŒïŒ-> group_positionã
ã·ã³ã°ã«ãæ€çŽ¢ããå Žåãæåã«ãã®èŸæžã§ã°ã«ãŒãã®äœçœ®ãæ€çŽ¢ãã次ã«ã°ã«ãŒããã¢ã³ããŒãããŠã¡ã¢ãªã§çŽæ¥æ€çŽ¢ããŸãã æäœå šäœã«ã¯2åã®èªã¿åããå¿ èŠã§ãã
ã°ã«ãŒãäœçœ®ã®ãã£ã¯ã·ã§ããªã¯ãã€ã³ããã¯ã¹èªäœãããæ°æ¡å°ãªãã¹ããŒã¹ããå¿ èŠãšããŸãããå€ãã®å ŽåãåçŽã«ã¡ã¢ãªã«ã¢ã³ããŒãã§ããŸãã ãããã£ãŠãèªã¿åãå€ã¯2ã€ã§ã¯ãªãã1ã€ã«ãªããŸãã åèšã OïŒ1ïŒ ã
3.ãã«ãŒã ãã£ã«ã¿ãŒ
é¢æ¥ã§ã¯ãåè£è ã¯OïŒn ^ 2ïŒãŸãã¯OïŒ2 ^ nïŒã§äžæã®ãœãªã¥ãŒã·ã§ã³ãçºè¡ããããšã§åé¡ã解決ããããšããããããŸãã ããããç§ãã¡ã¯æããªããšãããŸããã äžçã«OïŒ0ïŒããããŸããïŒããã¯è³ªåã§ããïŒ çµæãæåŸ ããã«è©ŠããŠã¿ãŸããã...
äž»é¡é åã«ç®ãåããŸãã çåŸããããã£ãã®ã§èªåã§äœåãæžããå ŽåããŸãã¯åã«ããã¹ãã§ã¯ãªããŽãããã£ãå Žåã垯ç¶ç±ç¹ã®éèŠãªéšåã¯äžæã§ããã玢åŒã«ã¯èŠã€ãããŸããã ãã«ãŒã ãã£ã«ã¿ãŒãªã©ã®ããŒã¿æ§é ã¯ãäžççã«ããç¥ãããŠããŸãã æ€çŽ¢ããåã«ãå±æ ¹æ¿ã確èªããŠãã ããã ã€ã³ããã¯ã¹ã«éçããªãå Žåã¯ãããã«èª¿ã¹ãããšã¯ã§ããŸããã
ãã«ãŒã ãã£ã«ã¿ãŒèªäœã¯éåžžã«åçŽã§ãããããªã¥ãŒã ã§ããã·ã¥é¢æ°ã®ãã¯ãã«ã䜿çšããããšã¯æå³ããããŸããã ãã«ãŒã ãã£ã«ã¿ãŒããã®èªã¿åãå€ã+1ããã ãã§ååã§ãã ããã¯ãã·ã³ã°ã«ãäžæã§ããããã£ã«ã¿ãŒã«èª€æ€åºããªãã£ãå Žåã«ãåŸç¶ã®ã¹ããŒãžãã-1ãŸãã¯-2ã®èªã¿åãå€ãæäŸããŸãã æãèŠãŠãã ããïŒ
ãã«ãŒã ãã£ã«ã¿ãŒãšã©ãŒã®ç¢ºçã¯æ§ç¯äžã«èšå®ãããæªç¥ã®éçã®ç¢ºçã¯çåŸã®èª å®ãã«ãã£ãŠæ±ºãŸããŸãã åçŽãªèšç®ã¯ã次ã®äŸåé¢ä¿ã«ãªããŸãã
- 人ã®èª å®ããä¿¡ããïŒã€ãŸããå®éã«ããã¥ã¡ã³ãããªãªãžãã«ã§ããïŒå Žåãæ€çŽ¢é床ã¯äœäžããŸãã
- ããã¥ã¡ã³ããæ確ã«ã€ãªãããŠããå Žåãæ€çŽ¢é床ã¯åäžããŸãããå€ãã®ã¡ã¢ãªãå¿ èŠã§ãã
åŠçãä¿¡é Œããããšã§ããä¿¡é Œãããæ€èšŒããããšããååããããå®è·µã¯ãã«ãŒã ãã£ã«ã¿ãŒããã®å©çãããããšã瀺ããŠããŸãã
ãã®ããŒã¿æ§é ã¯ã€ã³ããã¯ã¹èªäœãããå°ããããã£ãã·ã¥ã§ããã®ã§ãæè¯ã®å Žåããã£ã¹ã¯ã«ã¢ã¯ã»ã¹ããã«ã·ã³ã°ã«ãåé€ã§ããŸãã
4.éãå°Ÿ
ã»ãšãã©ã©ãã«ã§ãèŠããã垯ç¶ç±ç¹ããããŸãã ç·æ°ã«å ããå²åã¯ãããã§ãããæåã®ã¹ãããã§ã€ã³ããã¯ã¹ãäœæãããšãã2çªç®ã®ã¹ãããã§ã¯ãæ°åããã³æ°çŸMBã®ãµã€ãºã®ã°ã«ãŒããååŸã§ããŸãã ããããåå¥ã«èšæ¶ããæ€çŽ¢ã¯ãšãªããããã«ç Žæ£ããŸãã
ãã®äºçŽ°ãªæé ã2011幎ã«åããŠäœ¿çšããããšããã€ã³ããã¯ã¹ã®ãµã€ãºã¯ååã«ãªããæ€çŽ¢èªäœãå éãããŸããã
5.ãã®ä»ã®å°Ÿ
ããã§ããã·ã³ã°ã«ã«ã¯å€ãã®ããã¥ã¡ã³ããå«ããããšãã§ããŸãã ããã¯æ£åžžã§ãã æ°åãæ°çŸãæ°å...ããããã¡ã€ã³ã€ã³ããã¯ã¹å ã«ä¿æããããšã¯æ¡ç®ãåããªããªããã°ã«ãŒãã«ãåãŸãããããããã°ã«ãŒãäœçœ®ã®èŸæžã®ããªã¥ãŒã ãèšãã¿ãŸãã ããå¹ççãªã¹ãã¬ãŒãžãåããå¥ã®ã·ãŒã±ã³ã¹ã«é 眮ããŸãã çµ±èšã«ãããšããã®ãããªæ±ºå®ã¯æ£åœåããã以äžã®ãã®ã§ãã ããã«ãããŸããŸãªãããåäœã®ããã±ãŒãžã«ããããã£ã¹ã¯ã¢ã¯ã»ã¹ã®åæ°ãæžãããã€ã³ããã¯ã¹ãµã€ãºãæžããããšãã§ããŸãã
ãã®çµæãã¡ã³ããã³ã¹ã容æã«ããããã«ãããããã¹ãŠã®ã¬ã€ã€ãŒã1ã€ã®å€§ããªãã¡ã€ã«ïŒãã£ã³ã¯ïŒã«å°å·ããŸãã ãã®ãããªå±€ã10åãããŸãã ãã ããäžéšã¯æ€çŽ¢ã§äœ¿çšããããäžéšã¯éåžžã«å°ãããåžžã«ã¡ã¢ãªã«æ ŒçŽãããŸããäžéšã¯å¿ èŠã«å¿ããŠ/å¯èœãªéãã¢ã¯ãã£ãã«ãã£ãã·ã¥ãããŸãã
æŠéã§ã¯ãã»ãšãã©ã®å Žåãã·ã³ã°ã«ã®æ€çŽ¢ã¯ãã¡ã€ã«ã®1ã€ãŸãã¯2ã€ã®ã©ã³ãã ãªèªã¿åãã«ãªããŸãã ææªã®å Žåã3ã€ãå®è¡ããå¿ èŠããããŸãã ãã¹ãŠã®ã¬ã€ã€ãŒã¯ãå¹æçã«ïŒå Žåã«ãã£ãŠã¯ãããåäœã§ïŒåºå®é·ã®èŠçŽ ã®ããã¯ãããé åã§ãã ãããæ£èŠåã§ãã 解åäžã®æéã¯ãä¿ç®¡äžã®åèšããªã¥ãŒã ã®äŸ¡æ Œãšæ¯èŒããŠéèŠã§ã¯ãªãããã£ãã·ã¥æ©èœãåäžããŸãã
æ§ç¯æã«ã¯ãã¬ã€ã€ãŒã®ãµã€ãºã¯äž»ã«äºåã«èšç®ãããé çªã«æžã蟌ãŸããããããã®æé ã¯éåžžã«é«éã§ãã
ã©ã®ããã«ããŠããã«ãã©ãçããã®ã
2010 , . , . , .
ç»åãœãŒã¹ïŒ ãŠã£ãããã£ã¢
åœåãã€ã³ããã¯ã¹ã¯2ã€ã®éšåã§æ§æãããŠããŸãããäžèšã®å®æ°ãšãSQLãBDBããŸãã¯ç¬èªã®æŽæ°ãã°ã®åœ¹å²ãæã€äžæçãªéšåã§ãã ããšãã°ãæã«1åïŒå Žåã«ãã£ãŠã¯1幎ã«1åïŒãäžæçãªãã®ããœãŒãããããã£ã«ã¿ãªã³ã°ãããŠãã¡ã€ã³ã®ãã®ãšããŒãžãããããšããããŸãã çµæã¯çµ±äžããããã®ã§ããã2ã€ã®å€ããã®ã¯åé€ãããŸããã äžæçãªãã®ãRAMã«åãŸããªãå Žåãæé ã¯å€éšãœãŒããééããŸããã
ãã®æé ã¯ããªãé¢åã§ãã»ãããã¥ã¢ã«ã¢ãŒãã§éå§ãããã€ã³ããã¯ã¹ãã¡ã€ã«å šäœãæåããæžãçŽãå¿ èŠããããŸããã æ°çŸäžã®ããã¥ã¡ã³ãã®ããã«æ°çŸã®ã¬ãã€ãã®æžãæã-ãŸãããŸããŸãã®åã³ãç§ã¯ããªãã«èšããŸã...
SSD. , 31 SSD wcf- . , . , .
SSDãç·åŒµãããããã€ã³ããã¯ã¹ãããé »ç¹ã«æŽæ°ãããããã«ã2012幎ã«æ¬¡ã®ã¹ããŒã ã«åŸã£ãŠãã£ã³ã¯ã®ããã€ãã®ããŒã¹ã®ãã§ãŒã³ãé¢äžããŸããã
ããã§ã¯ãã€ã³ããã¯ã¹ã¯ãæåã®ãã®ãé€ããåãã¿ã€ãã®ãã£ã³ã¯ã®ãã§ãŒã³ã§æ§æãããŠããŸãã æåã®ã¢ããªã³ã¯ãRAMã«ã€ã³ããã¯ã¹ãããè¿œå å°çšãã°ã§ããã åŸç¶ã®ãã£ã³ã¯ã¯ãæåŸïŒãŒããã¡ã€ã³ãã«ãŒããªã©ïŒãŸã§ãµã€ãºïŒããã³çµéæéïŒãå¢å ããŸããã
ããã¥ã¡ã³ããè¿œå ãããšããæåã«ã¢ããªã³ã«æãç³ãŸããŸããã ãªãŒããŒãããŒãŸãã¯ä»ã®åºæºã«ãããæ°žç¶çãªãã£ã³ã¯ãæ§ç¯ãããŸããã å¿ èŠã«å¿ããŠãé£æ¥ããããã€ãã®ãã£ã³ã¯ã¯æ°ãããã®ã«ããŒãžãããå ã®ãã£ã³ã¯ã¯åé€ãããŸããã ããã¥ã¡ã³ãã®æŽæ°ãŸãã¯åé€ãåãããã«æ©èœããŸããã
ããŒãžåºæºããã§ãŒã³ã®é·ãããã€ãã¹ã¢ã«ãŽãªãºã ãåé€ãããã¢ã€ãã ãšæŽæ°ã®ã¢ã«ãŠã³ãã£ã³ã°ããã®ä»ã®ãã©ã¡ãŒã¿ãŒã調æŽãããŸããã ã¢ãããŒãèªäœã¯ãããã€ãã®åæ§ã®ã¿ã¹ã¯ã«é¢äžããŠãããã¯ãªãŒã³ãª.netäžã®å¥åã®å éšLSMãã¬ãŒã ã¯ãŒã¯ãšããŠåœ¢ã«ãªããŸããã åãé ãLevelDBã人æ°ãåããŸããã
ãã®å Žåã®LSMã¢ã«ãŽãªãºã ã«ã¯ãéåžžã«é©ããæ©èœããããŸãã
- ã€ã³ã¹ã¿ã³ãæ¿å ¥/åé€/æŽæ°ã
- æŽæ°äžã®SSDã®è² è·ã®è»œæžã
- ç°¡ç¥åããããã£ã³ã¯åœ¢åŒã
- å€ã/æ°ãããã£ã³ã¯ã®ã¿ã®éžæçæ€çŽ¢ã
- äºçŽ°ãªããã¯ã¢ãã
- éã¯ä»ã«äœãæãã§ããŸããã
- ...
äžè¬çã«ãèªå·±éçºã®ããã«èªè»¢è»ãçºæããããšã¯æã æçšã§ãã
ãã¯ãããã€ã¯ããããã®æé©å
ãããŠæåŸã«ã 察çäœã®ç§ãã¡ã.Netã§ïŒãããŠããã ãã§ãªãïŒãã®ãããªããšãè¡ãæ¹æ³ã«é¢ããæè¡çãªãã³ããå ±æããŸãã
å€ãã®å Žåããã¹ãŠãç¹å®ã®ããŒããŠã§ã¢ãããŒã¿ããŸãã¯äœ¿çšã¢ãŒãã«å€§ããäŸåããããšã«äºåã«æ³šæããŠãã ããã ããå Žæã§ããããåŸãCPUãã£ãã·ã¥ããé£ã³åºããå¥ã®å Žæã§-SATAã€ã³ã¿ãŒãã§ãŒã¹ã®åž¯åå¹ ã«ã¶ã€ãã£ãŠã3çªç®ã«-GCã§ãã³ã°ãå§ããŸãã ãããŠãç¹å®ã®ã·ã¹ãã ã³ãŒã«ã®å®è£ ã®éå¹çæ§ã®ã©ããã«ã
ç»åãœãŒã¹ïŒ ãŠã£ãããã£ã¢
ãã¡ã€ã«ãæäœãã
ãã¡ã€ã«ãžã®ã¢ã¯ã»ã¹ã®åé¡ã¯ãç§ãã¡ã ãã§ã¯ãããŸããã ãã©ãã€ããšã¯ãµãã€ãã®å€§ããªãã¡ã€ã«ãããããã®å®¹éã¯RAMã®éã®äœåã«ããªããŸãã ã¿ã¹ã¯ã¯ãåšå²ã«æ£ãã°ã£ãŠããæ°çŸäžåã®å°ããªã©ã³ãã å€ãèªã¿åãããšã§ãã ãããŠããããè¿
éãå¹ççãã€å®äŸ¡ã«è¡ãããã«ã çµã蟌ã¿ããã³ãããŒã¯ããããèããªããã°ãªããŸããã
ç°¡åãªãã®ããå§ããŸãããã 倧åãªãã€ããèªãã«ã¯ã次ã®ãã®ãå¿ èŠã§ãã
- ãã¡ã€ã«ãéãïŒæ°ããFileStreamïŒ;
- ç®çã®äœçœ®ã«ç§»åããŸãïŒäœçœ®ãŸãã¯ã·ãŒã¯ãå·®ãªãïŒã
- ç®çã®ãã€ãé åãèªã¿åããŸãïŒèªã¿åãïŒã
- ãã¡ã€ã«ãéããŸãïŒç Žæ£ïŒã
ãããŠãããã¯é·ããŠéå±ã ããã§ãã è©Šè¡é¯èª€ããã¬ãŒããç¹°ãè¿ãèžãããšã§ã次ã®ã¢ã¯ã·ã§ã³ã¢ã«ãŽãªãºã ãç¹å®ããŸããã
ã·ã³ã°ã«ãªãŒãã³ããã«ããªãŒã
ãã®ã·ãŒã±ã³ã¹ãé¡ã§è¡ããšããã£ã¹ã¯ãžã®ãã¹ãŠã®èŠæ±ã«å¯ŸããŠãããã«æ²ãããŸãã ãããã®åé ç®ã¯ãOSã«ãŒãã«ãžã®èŠæ±ã«ãªããŸãããããã¯é«äŸ¡ã§ãã
æããã«ããã¡ã€ã«ãäžåºŠéããããããäœçŸäžãã®ãã¹ãŠã®å€ãé çªã«èªã¿åãå¿ èŠããããŸãã
äœåãªãã®ã¯ãããŸãã
ãã¡ã€ã«ãµã€ãºããã®äžã®çŸåšã®äœçœ®ãååŸããããšãéåžžã«é£ããæäœã§ãã ãã¡ã€ã«ãå€æŽãããªãã£ãå Žåã§ãã
ãã¡ã€ã«ãµã€ãºãçŸåšã®äœçœ®ãååŸãããªã©ã®ã¯ãšãªã¯é¿ããŠãã ããã
Filestreampool
次ã æ®å¿µãªãããFileStreamã¯åºæ¬çã«ã·ã³ã°ã«ã¹ã¬ããã§ãã ãã¡ã€ã«ã䞊è¡ããŠèªã¿åãããå Žåã¯ãæ°ãããã¡ã€ã«ã¹ããªãŒã ãäœæ/éããå¿ èŠããããŸãã
aiosyncã®ãããªãã®ãäœæãããŸã§ãç¬èªã®èªè»¢è»ãçºæããå¿ èŠããããŸãã
ç§ã®ã¢ããã€ã¹ã¯ããã¡ã€ã«ããšã«ãã¡ã€ã«ã¹ããªãŒã ã®ããŒã«ãäœæããããšã§ãã ããã«ããããã¡ã€ã«ãéãããéãããããæéãç¡é§ã«ããããšããªããªããŸãã ãããŠããããThreadPoolãšçµã¿åãããŠãSSDã匷åãªãã«ãã¹ã¬ããã§megaIOPSãçºè¡ããããšãèæ ®ã«å ¥ãããš...ããã§ãããç§ãç解ããŠããŸãã
é ååäœ
次ã ã¹ãã¬ãŒãžããã€ã¹ïŒHDDãSSDãOptaneïŒããã³ãã¡ã€ã«ã·ã¹ãã ã¯ããããã¯ã¬ãã«ïŒã¯ã©ã¹ã¿ãŒãã»ã¯ã¿ãŒãå²ãåœãŠåäœïŒã®ãã¡ã€ã«ã§åäœããŸãã äžèŽããªãå ŽåããããŸãããçŸåšã§ã¯ã»ãšãã©ã®å Žå4096ãã€ãã§ãã SSDå ã®ãã®ãããª2ã€ã®ãããã¯ã®å¢çã§1ãŸãã¯2ãã€ããèªã¿åãããšã¯ããããã¯èªäœã®å éšãããçŽ1.5åé ããªããŸãã
æžç®ããèŠçŽ ã
ã¯ã©ã¹ã¿ãŒã»ã¯ã¿ãŒãããã¯ã®å¢çå ã«åãŸãããã«ãããŒã¿ãæŽçããå¿ èŠããããŸãã
ãããã¡ãªãã
次ã FileStreamã¯ããã©ã«ãã§4096ãã€ãã®ãããã¡ãŒã䜿çšããŸãã ãããŠæªããã¥ãŒã¹ã¯ãããããªãã«ã§ããªãããšã§ãã ãã ãããããã¡ã®ãµã€ãºãããå€ãã®ããŒã¿ãèªã¿èŸŒãã§ããå ŽåãåŸè ã¯ç¡èŠãããŸãã
ã©ã³ãã èªã¿åãã®å Žåã¯ããããã¡ãŒã1ãã€ãã«èšå®ãïŒããããå°ãªãå Žåã¯æ©èœããŸããïŒã䜿çšããªããšæ³å®ããŸãã
ãããã¡ã䜿çšããŸãã
ã©ã³ãã ãªèªã¿åãã«å ããŠãé 次èªã¿åãããããŸãã ããã§ãäžåºŠã«ãã¹ãŠãèªã¿ãããªãå Žåã¯ããããã¡ããã§ã«åœ¹ç«ã€ãããããŸããã ãã®èšäºããå§ããããšããå§ãããŸãã èšå®ãããããã¡ãµã€ãºã¯ããã¡ã€ã«ãHDDã«ãããSSDã«ãããã«ãã£ãŠç°ãªããŸãã æåã®ã±ãŒã¹ã§ã¯1MBãæé©ã§ã2çªç®ã®ã±ãŒã¹ã§ã¯æšæºã®4kBã§ååã§ãã èªã¿åãããããŒã¿é åã®ãµã€ãºããããã®å€ã«å¹æµããå Žåã¯ãã©ã³ãã èªã¿åãã®å Žåã®ããã«ããããã¡ãŒãã¹ãããããŠäžåºŠã«æžç®ããããšããå§ãããŸãã ãããã¡ã倧ãããšãé床ã¯åäžããŸããããGCã«ããããå§ããŸãã
ãã¡ã€ã«ã®å€§ããªéšåãé£ç¶ããŠèªã¿åããšãã¯ããããã¡ãHDDçšã«1MBãSSDçšã«4kBã«èšå®ããå¿ èŠããããŸãã ãŸããããã¯äŸåããŸãã
MMF vs FileStream
ãã®ã¡ã«ããºã ã¯.Net Framework v4.0以éã§å®è£ ãããããã2011幎ã«MemoryMappedFileã«ãã³ãããããŸããã æåã«ã圌ãã¯ãã«ãŒã ãã£ã«ã¿ãŒããã£ãã·ã¥ãããšãã«ããã䜿çšããŸãããããã¯ã4GBã®å¶éã®ããã«32ãããã¢ãŒãã§ã¯æ¢ã«äžäŸ¿ã§ããã ãããã64ãããã®äžçã«ç§»è¡ãããšãããã£ãšæ¬²ãããšæããŸããã æåã®ãã¹ãã¯å°è±¡çã§ããã ç¡æã®ãã£ãã·ã¥ãç°åžžãªé床ã䟿å©ãªæ§é èªã¿åãã€ã³ã¿ãŒãã§ã€ã¹ã ããããåé¡ããããŸããïŒ
- ãŸããå¥åŠãªããšã«ãé床ã ããŒã¿ããã§ã«ãã£ãã·ã¥ãããŠããå Žåã¯ããã¹ãŠåé¡ãããŸããã ããããããã§ãªãå Žåããã¡ã€ã«ãã1ãã€ããèªã¿åãã«ã¯ãéåžžã®èªã¿åããããã¯ããã«å€ãã®ããŒã¿ã®ããªããã¢ãããã䌎ããŸãã
- 第äºã«ãå¥åŠãªããšã«ãã¡ã¢ãªã å ç±ãããšãå ±æã¡ã¢ãªãå¢å ããworkingset-ããããè«ççã§ãã ãããããã®åŸãé£æ¥ããããã»ã¹ã¯ããŸãããŸãåäœããªããªããŸãã 圌ãã¯ã¹ã¯ããã«å ¥ãããOoMãã誀ã£ãŠèœã¡ãå¯èœæ§ããããŸãã æ²ããããªãRAMã®MMFãå ããããªã¥ãŒã ã¯å¶åŸ¡ã§ããŸããã ãŸããèªã¿åããã¡ã€ã«èªäœãã¡ã¢ãªãããæ°æ¡å€§ããå Žåã®ãã£ãã·ã¥ããã®å©çã¯ç¡æå³ã«ãªããŸãã
2çªç®ã®åé¡ã¯ãŸã 解決ã§ããŸããã ã€ã³ããã¯ã¹ãdockerãŸãã¯å°çšã®ä»®æ³ãã·ã³ã§æ©èœããå Žåã¯è¡šç€ºãããŸããã ããããé床ã®åé¡ã¯èŽåœçã§ããã
ãã®çµæãMMFã¯å®å šã«æŸæ£ãããŸããã å¯èœãªå Žåã¯ãæå®ãããåªå 床ãšå¶éã§æãé »ç¹ã«äœ¿çšãããã¬ã€ã€ãŒãã¡ã¢ãªã«ä¿æããæ瀺çãªåœ¢åŒã§åè¬ç©äž»çŸ©ã®ãã£ãã·ã³ã°ãè¡ããå§ããŸããã
ç»åãœãŒã¹ïŒ ãŠã£ãããã£ã¢
ããã/ãã€ã
äžçã¯1ãã€ãã§ã¯ãããŸããã æã ããããã¬ãã«ãŸã§äžããå¿ èŠããããŸãã
ããšãã°ã1å åã®éšåçã«é åºä»ããããçªå·ããããä¿åãšé »ç¹ãªèªã¿åãã«ç±å¿ã§ãããšããŸãã ããããã¹ãŠãã©ã®ããã«äœ¿çšããŸããïŒ
- ã·ã³ãã«ãªBinaryWriter.WriteïŒ -éããé ãã ãµã€ãºã¯éèŠã§ãã ã³ãŒã«ããªãŒãã£ã³ã°ã¯äž»ã«ãã¡ã€ã«ãµã€ãºã«äŸåããŸãã
- VarIntã®å¥ã®ããªãšãŒã·ã§ã³ã¯ïŒ -éããé ãã äžè²«æ§ãéèŠã§ãã ããªã¥ãŒã ã¯ããŒã¿ã«äŸåãå§ãããããäœçœ®æ±ºãã«ã¯è¿œå ã®ã¡ã¢ãªãå¿ èŠã§ãã
- ããããããã³ã°ïŒ -éããé ãã ããªãã¯ãã£ãšæ³šææ·±ãæãå¶åŸ¡ããªããã°ãªããŸããã
çæ³çãªè§£æ±ºçã¯ãããŸããããç¹å®ã®å Žåã32ãããããããŒã«ãæ ŒçŽããã®ã«å¿ èŠãªç¯å²ãŸã§ã®åçŽãªå§çž®ã¯ãVarIntããã12ïŒ ïŒãã¡ãããé£æ¥ãããã®ã®éãã®ã¿ãä¿åïŒç¯çŽããããã¯æ°åã§ãåºæ¬ãªãã·ã§ã³ã
å¥ã®äŸã ãã¡ã€ã«å ã®ããã€ãã®é åãžã®ãªã³ã¯ããããŸãã 64ãããã®ãªã³ã¯ããã©ãã€ãããšã®ãã¡ã€ã«ã ãã¹ãŠã倧äžå€«ã®ããã§ãã é åã«å€ãã®æ°åãããå Žåãããã°ãå°ãªãå ŽåããããŸãã ãã°ãã°å°ãã éåžžã«é »ç¹ã«ã ãã®åŸãé åå šäœãååŸããŠãªã³ã¯èªäœã«ä¿åããŸãã å©ç æ éã«æ¢±å ããŸãããå¿ããªãã§ãã ããã
æ§é ãå®å šã§ãªãããããåŠçããã€ã¯ããªãã
ããŠãä»ã®ãã€ã¯ãæé©åã ããã§ã¯ããã«ãŒãã®é åã®é·ããä¿åãã䟡å€ãããããŸãã¯ãé«éãforãŸãã¯foreachããšããå¹³å¡ã«ã€ããŠã¯æžããŸããã
2ã€ã®åçŽãªã«ãŒã«ããããããããé å®ããŸãïŒ1.ããã¹ãŠããã³ãããŒã¯ãããã2ããããå€ãã®ãã³ãããŒã¯ãã
æ§é äœ ã ã©ãã§ã䜿çšãããŸãã GCãåºè·ããªãã§ãã ããã ãŸããä»æ¥æµè¡ããŠãããããç¬èªã®è¶ é«éValueListããããŸãã
å®å šã§ãªã ã 䜿çšæã«ãã€ãã®é åãžã®mapitïŒããã³unmapïŒæ§é ãèš±å¯ããŸãã ãããã£ãŠãã·ãªã¢ã«åã®åå¥ã®æ段ã¯å¿ èŠãããŸããã 確ãã«ãããŒãã®ãã³çããšããã©ã°ã«ã¯çåããããŸããããããŸã§ã®ãšãã瀺ãããŠããŸããã ãŸããããã¯äŸåããŸãã
ãããåŠç ã å€ãã®èŠçŽ ãæ±ãã«ã¯ãããã¯/ã°ã«ãŒã/ãããã¯ã䜿çšããå¿ èŠããããŸãã ãã¡ã€ã«ã®èªã¿åã/æžã蟌ã¿ãæ©èœéã®è»¢éã å¥ã®åé¡ã¯ããããã®ããã¯ã®ãµã€ãºã§ãã éåžžãæé©ãªãµã€ãºãããããã®ãµã€ãºã¯1kBã8MBïŒCPUãã£ãã·ã¥ãµã€ãºãã¯ã©ã¹ã¿ãŒãµã€ãºãããŒãžãµã€ãºãªã©ïŒã®ç¯å²ã«ããããšããããããŸãã IEnumerable <byte>ãŸãã¯IEnumerable <byte [1024]>é¢æ°ã䜿çšããŠã¿ãŠãéããæããŠãã ããã
ããŒãªã³ã° ã ãæ°èŠããšæžããã³ã«ãåç«ã¯ã©ããã§æ»ã«ãŸãã äžåºŠæ°ãããã€ã[ 85000 ]-ãããŠãã©ã¯ã¿ãŒã¯ã¬ãã§ãŠã®ãã³ã«ä¹ã£ãã stackallocã䜿çšã§ããªãå Žåã¯ããªããžã§ã¯ãã®ããŒã«ãäœæããŠåå©çšããŸãã
ã€ã³ã©ã€ã³å ã 1ã€ã§ã¯ãªã2ã€ã®é¢æ°ãäœæãããšããã¹ãŠã10åé«éåã§ããŸããïŒ ã·ã³ãã«ã é¢æ°ïŒã¡ãœããïŒã®æ¬äœã®ãµã€ãºãå°ããã»ã©ãã€ã³ã©ã€ã³ã«ãªãå¯èœæ§ãé«ããªããŸãã æ®å¿µãªãããããããããã®äžçã§ã¯éšåçãªã€ã³ã©ã€ã³åãè¡ãæ¹æ³ããŸã ãªããããæåã®æ°è¡ãåŠçããåŸã«99ïŒ ã®ã±ãŒã¹ã§åºãŠãããããé¢æ°ããããæ®ãã®100è¡ãæ®ãã®1ïŒ ãåŠçããŠããããããå®å šã«åå²ããŸã2ïŒãŸãã¯3ïŒãéãå°Ÿãå¥ã®æ©èœã«éã³ãŸãã
ä»ã«äœïŒ
ã¹ãã³<T> ã ã¡ã¢ãª<T> -ææã§ãã ã³ãŒãã¯ããã·ã³ãã«ã«ãªããå°ãéããªããŸãã .Net Core v3.0ããã³Std v2.1ã®ãªãªãŒã¹ãåŸ ã£ãŠããŸãã éåžžãã¹ãã³ããµããŒãããªã.Net Std v2.0ã®ã«ãŒãã«ã
éåæ/åŸ æ© -ä»ã®ãšããç©è°ãéžãã æãåçŽãªã©ã³ãã èªã¿åããã³ãããŒã¯ã¯ãCPUæ¶è²»éãå®éã«äœäžããŠããããšã瀺ããŸããããèªã¿åãé床ãäœäžããŠããŸãã èŠãªããã°ãªããŸããã ã€ã³ããã¯ã¹ã¯ãŸã 䜿çšããŠããŸããã
ãããã«
ç§ã®é éæ§ããããã€ãã®æ±ºå®ã®çŸãããç解ããåã³ãããªãã«äžããããšãé¡ã£ãŠããŸãã ã€ã³ããã¯ã¹ãæ¬åœã«æ°ã«å ¥ã£ãŠããŸãã ããã¯å¹ççã§çŸããã³ãŒãã§ãããçŽ æŽãããåããããŸãã äœæ¥ã®éèŠãªå Žæã§ããã·ã¹ãã ã®ã³ã¢ã«ãããé«åºŠã«å°éåããããœãªã¥ãŒã·ã§ã³ã¯ãäžè¬çãªãœãªã¥ãŒã·ã§ã³ãããåªããŠããŸãã ããŒãžã§ã³ç®¡çã·ã¹ãã ã¯ãC ++ã³ãŒãã§ã®ã¢ã»ã³ãã©ã®æ¿å ¥ãèšæ¶ããŠããŸãã çŸåšã4ã€ã®ãã©ã¹ããããŸã-çŽç²ãªCïŒã®ã¿ã.Netã®ã¿ã§ãã ãã®äžã«ãç§ãã¡ã¯æãè€éãªæ€çŽ¢ã¢ã«ãŽãªãºã ã§ãããæžãããŸã£ããåŸæããŸããã Dockerãžã®ç§»è¡ã§ãã.Net Coreã®åºçŸã«ãããDevOpsã®ããæããæªæ¥ãžã®éãããç°¡åãã€æ確ã«ãªããŸããã ä»åŸã¯ããœãªã¥ãŒã·ã§ã³ã®æå¹æ§ãšçŸãããæãªãããšãªããåçãªã·ã£ãŒãã£ã³ã°ãšã¬ããªã±ãŒã·ã§ã³ã®åé¡ã解決ããŸãã
æåŸãŸã§èªãã§ãããã¿ããªã«æè¬ããŸãã ãã¹ãŠã®äžäžèŽããã³ãã®ä»ã®ççŸã«ã€ããŠã¯ãã³ã¡ã³ããèšå ¥ããŠãã ããã ã³ã¡ã³ãã§ã®åççãªã¢ããã€ã¹ãšåè«ãæè¿ããŸãã