
The AV在线观看2025¹Ù·½×îа汾ÏÂÔØ
ÏêÇé½éÉÜ
The AV在线观看×°ÖðüÏÂÔØÊÇÒ»¿îÄ£Äâ¾ÓªÕ½ÂÔÓÎÏ·£¬¸Ã°æ±¾Íæ¼Ò¿ÉÒÔÖ±½Óͨ¹ý°²×¿Ä£ÄâÆ÷ÔÚµçÄÔÉÏ×°ÖÃÌåÑé¡£¸ÃÓÎÏ·½ÓÄÉΨÃÀµÄˮī»·ç£¬½«Öйú·çÔªËØÈÚÈëÓÎÏ·³¡¾°£¬ÎªÍæ¼Ò´øÀ´¼«ÖµÄÊÓ¾õÏíÊÜ£¬ÈÃÄúÌÕ×íÆäÖУ¬¸ÐÊÜw3u7903ejky2ywlsÖ®ÃÀ¡£ÔÚÓÎÏ·ÖУ¬Íæ¼Ò½«ÊÎÑÝÒ»Î»×æÊ¦£¬¿ª×ÚÁ¢ÅÉ£¬ÅàÑøÒ»ÖÚÓÐȤµÄÃÅÉú£¬×ÊÖúËûÃǶɽٳÉÏÉ¡£Ã¿Î»ÃÅÉú¶¼ÓµÓÐÆæÌصÄÃü¸ñºÍÊôÐÔ£¬¸öÐÔåÄÒ죬ÈÃÄúÌåÑéµ½Ç§Ææ°Ù¹ÖµÄÐÞÏÉÉú»î¡£
Óë´Ëͬʱ£¬The AV在线观看µçÊÓ°æÏÂÔØ»¹ÓµÓÐÆæÌصĹһú»úÖÆ£¬Äú¿ÉÒÔ½«ÓÎÏ·°²ÅÅÔÚºǫ́£¬½â·ÅË«ÊÖ£¬ÈÃÃÅÉúÃÇ×Ô¶¯ÐÞÁ¶¡¢¶É½Ù£¬ÌùÐĺǻ¤ÄúµÄÐÞÏÉÃÅÅÉ¡£×ÚÃŵزú½¨ÉèÒ²ÊÇÓÎÏ·µÄÖØÒªÄÚÈÝ£¬Äú¿ÉÒÔ×ÔÓɰڷţ¬´òÔìÊôÓÚ×Ô¼ºµÄÐÞÏÉ×ÚÃÅ£¬´´Á¢ÏÉÃÅÈ˵ÄÀíÏë¼ÒÔ°¡£´Óɽº£ÒìÊÞµ½Ò»Ê¯Ò»Ä¾£¬Ëæ´¦³äÂú׏ÅÔÏÏɷ磬ÈÃÄúËÆºõÖÃÉíÓÚÐÞÏÉС˵°ãµÄÏɾ³Ö®ÖС£
OpenAI¾ÃÎ¥·¢ÁËÆª¡¸Õý¾¡¹ÂÛÎÄ£ºÏßÐԽṹʵÏÖ¸ßЧÕÅÁ¿ÅÌËãÌØÉ«
»úе֮Ðı¨µÀ
±à¼£ºPanda
OpenAI ·¢ÂÛÎĵįµÂÊÊÇÔ½À´Ô½µÍÁË¡£
Èç¹ûÄã¿´µ½ÁËÒ»·ÝÀ´×Ô OpenAI µÄРPDF Îļþ£¬ÄÇ´ó¶¼Ò²ÊÇÐÂÄ£Ð͵Äϵͳ¿¨»òÏà¹ØÔö²¹Îļþ»ò»ù×¼²âÊÔ£¬ºÜÉÙÓÐеÄÑо¿ÂÛÎÄ¡£
ÖÁÓÚÔÒòÂÈøù«Ë¾×Ô¼ÒµÄ ChatGPT À´Ëµ°É£º¡¸½ØÖÁĿǰ£¬OpenAI ÔÚ 2025 ÄêÔÚ arXiv ÉϹûÕæÐû²¼µÄÂÛÎÄÊýÁ¿Ïà¶Ô½ÏÉÙ£¬¿ÉÄÜ·´Ó¦ÁËÆä¶ÔÑо¿½á¹û¹ûÕæÕ½ÂԵĽ÷É÷̬¶È£¬¿ÉÄܳöÓÚÉÌÒµ±£ÃÜ»òÄþ¾²¿¼ÂÇ¡£¡¹
²»¹ý½üÈÕ£¬OpenAI ҲȷʵÐû²¼ÁËÒ»·ÝÍêÈ«ÓÉ×Ô¼ºÈ˼ÓÈëµÄ¡¢Êµ´òʵµÄÑо¿ÂÛÎÄ£¬ÆäÖÐÌá³öÁËÒ»ÖÖÓÃÓÚ¸ßЧÕÅÁ¿Ó³ÉäµÄͳһ´úÊý¿ò¼ÜLinear Layouts¡£ÕâÊÇÒ»ÖÖʹÓöþÔªÏßÐÔ´úÊý¶ø·Ç±ÈÌØÌåÏÖ£¨bit representation£©µÄÕÅÁ¿½á¹¹µÄͨÓôúÊýÐÎʽ£¬½â¾öÁË Triton µÈÉî¶Èѧϰ±àÒëÆ÷Öкã¾Ã±£´æµÄÄÑÌâ¡£
ÂÛÎÄÌâÄ¿£ºLinear Layouts: Robust Code Generation of Efficient Tensor Computation Using ?ÂÛÎĵص㣺https://arxiv.org/pdf/2505.23819.pdf
ÒªÀí½âÕâÏîÑо¿µÄÒâÒ壬Ê×ÏÈÐèÒªÏÈÀí½âÒ»ÏÂʲôÊÇÕÅÁ¿½á¹¹£¨tensor layouts£©
¼òµ¥À´Ëµ£ºÕÅÁ¿½á¹¹ = Âß¼ÕÅÁ¿ÓëÓ²¼þ×ÊÔ´£¨ÀýÈçÄÚ´æ¡¢Ï̡߳¢ÏòÁ¿µ¥Î»£©Ö®¼äµÄÓ³Éä¹ØÏµ¡£ÏÂͼ¸ø³öÁËÁ½¸ö½á¹¹Ê¾Àý¡£
¹ØÓÚÏÖ´úÉî¶ÈѧϰÊÂÇé¸ºÔØ¶øÑÔ£¬ËùÐèÒªµÄÕÅÁ¿½á¹¹ÐèÒªÂú×㼸¸öÒªÇó£º
¸ßЧ£¨ÎªÁËÐÔÄÜ£©¡£Áé»î£¨ÒÔÖ§³Ö¶àÖÖËã×Ó£©¡£¿É×éºÏ£¨ÎªÁ˱任ºÍÓÅ»¯£©¡£
È»¶ø£¬Ä¿½ñµÄ½á¹¹ÏµÍ³È´ÄÑÒÔ³ä·ÖÂú×ãÕâЩÐèÇ󣬶øÊÇÍùÍù£º
ÐèҪƾ¾Ýʵ¼ÊÐèÇóÉè¼Æ£¬²¢ÇÒÍùÍùÊÇÓ²±àÂëµÄ£¨ÐèÒªÊÖ¶¯±àд¹æÔò£©¡£²»¿ÉÀ©Õ¹£¨Ã¿Ò»¶Ô½á¹¹¶¼ÐèÒª¶þ´Î×éºÏ£©¡£ÈÝÒ×ÍÉ»¯£¬ÓÈÆäÊÇÔÚÏñ Triton ÕâÑùµÄµÍ²ã¼¶µÄºó¶ËÖÐ ¡ª¡ª ½ØÖÁĿǰ£¬Triton µÄ GitHub ¿âÖÐÌá½»µÄ 12% µÄ Bug Óë½á¹¹Óйء£
ÁíÍ⣬Éî¶ÈѧϰӲ¼þ£¨Èç GPU£©µÄÈÕÒæÅÓ´óÒ²µ¼ÖÂÕÅÁ¿½á¹¹ÈÕÒæÅÓ´ó¡£
ÀýÈ磬ΪÁËʵÏÖ¸ßЧµÄ¾ØÕó³Ë·¨£¬Ó¢Î°´ïÔÚ Ampere¡¢Hopper ºÍ Blackwell µÈ²î±ð´ú¼ÊµÄ GPU ÉϽÓÄÉÁ˲î±ðµÄʹÓà Tensor Core µÄ½á¹¹£¬²¢ÇÒÿÖֽṹÔÚʹÓòî±ðÊý¾ÝÀàÐÍʱ¶¼Óвî±ðµÄ±äÌå¡£AMD ºÍÓ¢ÌØ¶ûµÈÆäËü GPU ¹©Ó¦ÉÌÔÚÀûÓÃÆäÀàËÆ Tensor Core µÄ¼¼Êõ½øÐмÓËÙʱ£¬Ò²Ê¹ÓÃÁ˲î±ðµÄ½á¹¹¡£Òò´Ë£¬Ó²¼þ¼Ü¹¹µÄ¿ìËÙÉú³¤ºÍ¶àÑù»¯µÄÉî¶ÈѧϰģÐÍÐèÒªÒ»ÖÖеÄÕÅÁ¿½á¹¹½¨Ä£ÒªÁì¡£
Ϊ´Ë£¬ÐèÒª½â¾öһЩ¼¼ÊõÄÑÌ⣺
ÔÚ½«ÕÅÁ¿Ó³Éäµ½Ó²¼þ×ÊÔ´·½Ã棬ÐèÒªÒ»ÖÖͨÓÃÇÒ¿É×éºÏµÄÌåÏÖÒªÁì¡£½á¹¹×ª»»Ó¦¸ÃÓÃͳһµÄÐÎʽÀ´±í´ï£¬ÉõÖÁÐèÒª°üÀ¨ÖîÈçÊý¾Ý½»»»£¨data swizzling£©µÈÅÓ´ó±ä»»¡£ÕâÖÖÌåÏÖ±ØÐëÓë³õ¼¶Ó²¼þÓÅ»¯Î޷켯³É£¬ÒÔÈ·±£¸ßЧµÄÊý¾Ý»á¼ûºÍÅÌËã¡£
²»¹ý£¬ÔÚ½éÉÜ OpenAI ÕâÆªÂÛÎĵÄТ¾´Ö®Ç°£¬ÎÒÃÇÐèÒªÏÈÁ˽âһЩ»ù´¡¿´·¨¡£
Ïà¹ØÅ侰֪ʶ
GPU ¼Ü¹¹
ÔÚÉè¼ÆÉÏ£¬ÏÖ´ú GPU µÄÄ¿±êÊÇͨ¹ý°üÀ¨¶à²ãÓ²¼þ×ÊÔ´µÄ·Ö²ãÖ´ÐÐÄ£ÐÍÀ´³ä·ÖÀûÓò¢ÐÐÐÔ¡£
ÆäÒªº¦Ö´Ðе¥Î»°üÀ¨Ð×÷Ïß³ÌÕóÁÐ (CTA)¡¢Warp ºÍÏ̡߳£Ã¿¸ö GPU Ï̶߳¼¿ÉÒÔ»á¼û˽ÓмĴæÆ÷ ¡ª¡ª ÕâЩ¼Ä´æÆ÷Ìṩ×îµÍÑӳٵĴ洢¿Õ¼ä£¬µ«ÈÝÁ¿ÓÐÏÞ¡£Í¨ÀýÖ¸Áî¿ÉÒÔÓɸ÷¸öÏ̶߳ÀÁ¢Ö´ÐС£È»¶ø£¬Ä³Ð©ÌØÊ⹦Чµ¥Î»±ØÐëÔÚ¸ü¸ßµÄÁ£¶È¼¶±ðÉÏÖ´ÐС£
ÀýÈ磬Ӣΰ´ïµÄ mma£¨¾ØÕó³Ë·¨ÀÛ¼Ó£©Ö¸ÁîÀûÓà Tensor Core µÄ·½·¨ÊDz¢ÐÐÖ´ÐÐÓɸ÷¸ö Warp ·¢³öµÄ¶à¸ö³Ë¼ÓÔËËã¡£¶ø wgmma£¨Warp ×龨Õó³Ë·¨ÀÛ¼Ó£©µÈ¸ß¼¶±äÌåÔòÊÇͨ¹ýÔÚ¶à¸ö Warp ÉÏͬʱִÐоØÕó³Ë·¨¶ø¶ÔÕâЩ¹¦Ð§½øÐÐÁËÀ©Õ¹¡£AMD Ò²ÒýÈëÁËÀàËÆµÄÔÓÀýÈç mfma£¨¾ØÕóÈںϳ˼ӣ©Ö¸Áî¡£
Çë×¢Ò⣬ÕâЩָÁîÒªÇóÊý¾ÝÂþÑÜÔÚÏß³ÌºÍ Warp Ö®¼ä£¬»òÕßÒÔÌØÊâ½á¹¹×¤ÁôÔÚ¹²ÏíÄÚ´æ»òÌØÊâÄڴ浥루ÀýÈç Blackwell É쵀 Tensor Memory£©ÖУ¬²Å»ª±¬·¢ÕýÈ·µÄ½á¹û¡£
È»¶ø£¬ÕâЩ½á¹¹Í¨³£²»»áΪ¼ÓÔØ / ´æ´¢µÈÆäËû²Ù×÷´øÀ´×î¼ÑÐÔÄÜ£¬²¢ÇÒ²¢·Ç×ÜÊÇ¿ÉÒÔʹÓÃÌØ¶¨Ö¸ÁÊý¾ÝÖ±½Ó´ÓÈ«¾ÖÄÚ´æ¸´ÖÆµ½ÌØÊâÄڴ浥λ¡£
Òò´Ë£¬Í¨³£±ØÐë¶ÔÊý¾Ý½øÐÐÖØÐÂÅÅÁУ¬ÒԱ㽫ÓÃÓÚÄÚ´æ»á¼ûµÄ½á¹¹×ª»»ÎªÅÌË㵥λƫºÃµÄ½á¹¹¡£
¼ò¶øÑÔÖ®£¬ÒªÊµÏÖ·åÖµÐÔÄÜ£¬²»µ«ÐèÒªÀûÓÃÕâЩרÓõ¥Î»£¬»¹ÐèÒª¾«ÐÄÉè¼ÆÕÅÁ¿½á¹¹ºÍת»»¡£
Triton ÓïÑԺͱàÒëÆ÷
Triton ÊÇÒ»ÖÖÀàËÆÓÚ Python µÄÓÃÓÚÌØ¶¨ÁìÓòµÄÓïÑÔ£¬ÆäÉè¼ÆÄ¿±êÊÇÌṩÓÃÓÚ±àд¸ßÐÔÄÜÉî¶ÈѧϰÔÓïµÄÁé»î½Ó¿Ú¡£Triton µÄ±àÒëÆ÷ºó¶ËʹÓÃÁË MLIR£¬Ö§³Ö¶àÌõÀíÁýͳ±í´ï¡£
¾¿Æä½¹µã£¬Triton ÄÚºË×ñѵ¥³ÌÐò´ó¶¼¾Ý (SPMD) Ä£ÐÍ£¬ÆäÖÐÅÌËã±»»®·ÖΪ¶à¸öÁýͳµÄ Triton ³ÌÐòʵÀý¡£ÕâÖÖÉè¼ÆÔÊÐí¿ª·¢ÕßÖ÷Òª¹Ø×¢ CTA ¼¶±ðµÄ²¢ÐÐÐÔ¼´¿É¡£ÔÚ Triton ÖУ¬¡¸ÕÅÁ¿¡¹Ò»´ÊÖ¸µÄÊÇ´ÓÔʼ PyTorch ÕÅÁ¿ÖÐÌáÈ¡µÄ¿é£¬ËüÃÇÓÃ×÷ GPU ºËµÄÊäÈëºÍÊä³ö¡£
ÔÚ±àÒëÀú³ÌÖУ¬Triton µÄ Python ´úÂëÊ×Ïȱ»·Òë³É Triton ·½ÑÔ (tt)£¬È»ºó½øÒ»²½·Òë³É TritonGPU ·½ÑÔ (ttg)¡£ÔÚ´ËÀú³ÌÖУ¬Ã¿¸öÕÅÁ¿¶¼ÓëÌØ¶¨µÄ½á¹¹Ïà¹ØÁª£¬ÒÔ³ä·ÖÀûÓÃÏÖ´ú GPU ÉÏ¿ÉÓõÄÓ²¼þ¹¦Ð§µ¥Î»¡£ÀýÈ磬µ±Óöµ½ dot ÀàËã×Ó£¨ÀýÈç tt.dot ºÍ tt.dot_scaled£©Ê±£¬»á½ÓÄÉ mma ½á¹¹²¢Ê¹Óà Tensor Core ºÍÀàËÆµÄµ¥Î»¡£
¹Å°å½á¹¹
ͼ 2 ÁгöÁË Triton ÖÐËùÓпÉÓõĽṹ¡£
ÔÚ×î¸ß²ã¼¶£¬½á¹¹·ÖΪÂþÑÜʽ£¨Distributed£©½á¹¹ºÍÄڴ棨£¨Memory£©½á¹¹¡£Ç°ÕßÊÇÖ¸ÕÅÁ¿ÔªËØÂþÑÜÔÚ²î±ðµÄÖ´Ðе¥Î»ÖУ¬´ËºóÕßÊÇÖ¸ÕÅÁ¿ÔªËØ´æ´¢ÔÚÌØ¶¨µÄÌØÊâÄÚ´æÖС£
ÂþÑÜʽ½á¹¹ÓֿɽøÒ»²½·ÖΪ Blocked¡¢Sliced¡¢MMA ºÍ MMA Input ½á¹¹µÈÀàÐÍ£¬¶øÄÚ´æ½á¹¹ÓֿɽøÒ»²½·ÖΪ Unswizzled ºÍ Swizzled ½á¹¹¡£
Blocked ½á¹¹Í¨³£ÓÃÓÚÁ¬ÐøµÄÄÚ´æ»á¼û¡£MMA ºÍ MMA ÊäÈë½á¹¹ÓÃÓÚ¾ØÕó³Ë·¨ÔËË㣨ÀýÈç tt.dot£©µÄÊä³öºÍÊäÈë¡£MMA ½á¹¹¿ÉÒÔÆ¾¾ÝÆäÓ³Éäµ½µÄÓ²¼þÖ¸Áî½øÒ»²½·ÖÀ࣬ÀýÈçӢΰ´ï GPU É쵀 mma ºÍ wgmma£¬»ò AMD GPU É쵀 mfma¡£Sliced ½á¹¹ÊÇ´ÓÆä¸¸½á¹¹ÖÐÌáȡһ¸öά¶È£¬ÓÃ×÷¹ã²¥»òij¸ö¹éÔ¼ÔËËãµÄÊä³ö¡£
¹Å°å Triton ½á¹¹ÏµÍ³ÒªÇóÿ¸ö½á¹¹½ç˵×Ô¼ºµÄ½Ó¿ÚÒªÁ죬ÀýÈçÿ¸öÏ̵߳ÄÔªËØÊýÁ¿ºÍÁ¬ÐøÔªËصÄÊýÁ¿¡£±ðµÄ£¬±ØÐëΪÿ¸ö½á¹¹ÏÔʽʵÏÖ¶ÔÕÅÁ¿ÔªËصÄË÷ÒýÒÔ¼°½á¹¹Ö®¼äµÄת»»¡£ÕâÖÖÒªÁìµ¼Ö½ṹ½á¹¹ºÍת»»³£·ºÆð bug¡£
Linear Layouts£¨ÏßÐԽṹ£©
ÏÂÃæ½«¼òµ¥½éÉÜÏßÐԽṹµÄ½ç˵¡¢Ò»Ð©»ù±¾µÄÏßÐԽṹËã×Ó¡¢´´Á¢ÖÖÖÖ Triton ½á¹¹ÒÔ×÷ΪÏßÐԽṹʵÀý£¬ÒÔ¼°Ó¦ÓÃÓÚ Triton µÄͨÓýṹÒýÇæ¡£
Ò»¸öʾÀý
ÔÚ GPU ±à³ÌÖУ¬´ó´ó¶¼²ÎÊý¶¼ÊÇ 2 µÄÃÝ£ºÒ»¸ö Warp ÓÉ 32 »ò 64 ¸öÏß³Ì×é³É£¬Ò»¸ö Warp ×é°üÀ¨ 4 ¸ö Warp£¬¾ØÕó³Ë·¨ÄÚÁªº¯Êý£¨ÀýÈç mma ºÍ wgmma£©ÒªÇó Tile ³ß´çΪ 16 ¡Á £¬ÆäÖÐ ¡Ý 1¡£
±ðµÄ£¬ÔÚ Triton µÄ±à³ÌÄ£ÐÍÖУ¬ÕÅÁ¿µÄά¶ÈÒÔ¼°Óëÿ¸öÕÅÁ¿Ïà¹ØµÄ½á¹¹×Ó²¿·Ö£¨ÀýÈçÿ¸öÏ̵߳ļĴæÆ÷ºÍÏß³ÌÊýÁ¿£©¶¼±»ÏÞÖÆÎª 2 µÄÃÝ¡£ÔÚͼ 1 ÖУ¬½á¹¹ A ÓÐÒ»¸ö 16 ¡Á 16 µÄÕÅÁ¿£¬ÆäʹÓÃÁ˶à¸ö 2 ¡Á 2 µÄ¼Ä´æÆ÷¡¢4 ¡Á 8 µÄÏß³ÌºÍ 2 ¡Á 1 µÄ Warp¡£
ÓÉÓÚÕâЩÁ¿¶¼ÊÇ 2 µÄÃÝ£¬Òò´ËʹÓÃÆä×ø±êµÄ±ÈÌØÌåÏÖ£¬¿ÉÒÔÖ±¹ÛµØ¿ÉÊÓ»¯½á¹¹ A ÖÐÔªËØµÄÂþÑÜ£¨Èçͼ 1 Ëùʾ£©¡£ËùÓÐÏ̵߳ļĴæÆ÷ 0 (_0) ¶¼Î»ÓÚ×ø±ê (, )£¬ÆäÖÐ ºÍ µÄ×îºó¼¸Î»£¨bit£©¾ùΪ 0¡£ÀýÈ磬Ïß³Ì _1 µÄ _0 λÓÚ (0, 2) = (000, 010)¡£×÷Ϊ±ÈÕÕ£¬_1 ÔªËØµÄ×ø±êÖУ¬ µÄ×îºóһλʼÖÕΪ 0£¬¶ø µÄ×îºóһλʼÖÕΪ 1¡£ÀýÈ磬_9 µÄ _1 λÓÚ (2, 3) = (010, 011)¡£
±ðµÄ£¬¹ØÓÚÈκÎżÊýÏß³Ì _£¬ µÄ×îºóһλÓë _0 ÖÐ µÄµ¹ÊýµÚ¶þλƥÅ䣬 µÄµ¹ÊýµÚ¶þλÓë _0 ÖÐ µÄµ¹ÊýµÚÈýλƥÅä¡£ÀýÈ磬_10 = _01010 µÄ _0 λÓÚ (2, 4) = (010, 0100)¡£ÕâÖÖϵͳÐÔ¶ÔÆëÁ¬Ðø±£´æ£¬±êÃ÷¶þ´ÎÃݽṹ×ãÒÔÇåÎúµØ¾ö¶¨ÁËÿ¸öÏß³ÌÔªËØµÄÂþÑÜ¡£
×ÛÉÏËùÊö£¬¼ÙÉèÒ»¸ö¾ÞϸΪ 8 µÄÏòÁ¿ ÌåÏÖÒ»¸ö Warp ÖÐÏ̵߳ÄÒ»¸öÔªËØ£¬ÆäÖÐǰ 2 λÌåÏּĴæÆ÷ (Reg)£¬½ÓÏÂÀ´µÄ 5 λÌåÏÖÏß³Ì (Thr)£¬×îºóһλÔòÌåÏÖ Warp (Wrp)£¬Ôò¿ÉÒÔÈç´Ë½ç˵½á¹¹ £º
µ±ÐèÒª´ÓÂß¼ÕÅÁ¿µÄ×ø±êÖлָ´Ó²¼þË÷Òýʱ£¬ÐèҪʹÓÃÇóÄæÔËËã¡£
¶ÔÏßÐԽṹµÄ¸üÏêϸÍ걸ÐÔ˵Ã÷Çë»á¼ûÔÂÛÎÄ£¬ÆäÖÐÉæ¼°µ½ËµÃ÷·Ö¿é½á¹¹¡¢mma ºÍ wgmma µÄÊäÈëºÍÊä³ö½á¹¹¡¢ÏßÐԽṹµÄ slice¡¢Ã¿¸öÂþÑÜʽ½á¹¹¡¢MMA swizzled ½á¹¹¡¢ÄÚ´æ½á¹¹¶¼ÊÇÏßÐԽṹ¡£ÁíÍ⣬OpenAI Ò²ÔÚ Triton ˵Ã÷ÁËÈçºÎʵÏֽṹת»»ÒÔ¼°ÐÎ×´²Ù×÷¡£
²»µ«Èç´Ë£¬OpenAI ÌåÏÖ£¬ÏßÐԽṹΪÔÚÓïÑÔǰ¶ËºÍ±àÒëÆ÷ºó¶Ë¿ª·¢Ëã·¨ÌṩÁ˽ṹ»¯µÄ»ù´¡¡£ËûÃÇÒ²ÔÚÂÛÎÄÖиø³öÁËһЩҪº¦Ê¾Àý£¬ÕâÀï¾Í²»¹ý¶àÕ¹¿ª¡£½ÓÏÂÀ´¼òµ¥¿´¿´ÐÂÌá³öµÄÏßÐԽṹµÄʵ¼ÊÌåÏÖ¡£
ÆÀ¹À
OpenAI ½«ÓÅ»¯°æ Triton£¨¼¯³ÉÁË»ùÓÚÏßÐԽṹµÄÓÅ»¯£¬¼´ Triton-Linear£©Óëδ¼¯³ÉÕâЩÓÅ»¯µÄ»ù×¼ Triton ½øÐÐÁ˱Ƚϡ£Triton ºÍ TritonLinear Ö®¼äµÄÖ÷񻂿±ðÈçÏ£º
Triton ʹÓùŰåµÄÊý¾Ý½á¹¹£¬²»Ö§³ÖÈÎÒâÂþÑÜʽ½á¹¹µÄʵÓóÌÐò»òËüÃÇÖ®¼äµÄת»»£¬Òò´ËÈÝÒ×·ºÆð bug¡£Triton δ½ÓÄÉÂÛÎÄÖÐÃèÊöµÄÓÅ»¯´úÂëÉú³É¡£ÀýÈ磬½á¹¹×ª»»Ê¼ÖÕͨ¹ý¹²ÏíÄÚ´æ½øÐУ¬¶Ô¸ßЧӲ¼þÔÓïµÄʹÓÃÓÐÏÞ¡£
¼ÓÈëÆÀ¹ÀµÄÓ²¼þƽ̨¼û±í 1¡£
ΪÁË±È½Ï Triton ºÍ Triton-Linear µÄÐÔÄÜ£¬¸ÃÍŶӹ¹½¨ÁËһЩºÏ³É΢»ù×¼À´½øÐвâÊÔ£¬Õâ·½ÃæµÄ½á¹ûÇë»á¼ûÔÂÛÎļì²ì¡£ÕâÀï½ö¿´¿´ËüÃÇÔÚʵ¼Ê»ù×¼²âÊÔÖÐÌåÏÖ¡£
ÔÚÈý¸ö²î±ðµÄƽ̨ÉÏ£¬OpenAI ÔËÐÐÁË TritonBench ÖÐµÄ 18 ¸ö»ù×¼²âÊÔ¡£Í¼ 7¡¢Í¼ 8 ºÍͼ 9 ÖÐչʾÁË Triton-Linear ÔÚÈý¸öƽ̨ÉϵÄÐÔÄÜÌáÉý¡£
ÓÉÓÚÿ¸ö»ù×¼²âÊÔ°üÀ¨¶à¸öÊäÈ룬×Ü¼Æ 420 ¸ö°¸Àý£¬Òò´ËËûÃÇʹÓÃÁËÎó²îÏߣ¨error bars£©À´ÌåÏÖÿ¸ö»ù×¼²âÊÔµÄ×îСºÍ×î´ó¼ÓËÙ¡£
ÐèҪעÒâµÄÊÇ£¬ÓÉÓÚÓ²¼þÏÞÖÆ£¬²¢·ÇËùÓлù×¼²âÊÔ¶¼ÊÊÓÃÓÚÿ¸öƽ̨¡£ÀýÈ磬ijЩ»ù×¼²âÊÔÐèÒª½öÔÚ GH200 ÉϲÅÓеĴóÐ͹²ÏíÄڴ棬¶øÒ»Ð©ºËʹÓõÄÕÅÁ¿ÃèÊö·ûÒÀÀµÓÚ TMA ÒýÇæ£¬¶ø RTX4090 ºÍ MI250 ÉϾù²»Ö§³Ö TMA ÒýÇæ¡£
¿ÉÒÔ¿´µ½£¬ÔÚ GH200 ÉÏ£¬ËûÃÇʵÏÖÁË 0.92 ±¶µ½ 1.57 ±¶²»µÈµÄ¼ÓËÙ£¬ËùÓлù×¼²âÊÔµÄÆ½¾ù¼ÓËÙ¾ùÁè¼Ý 1.0 ±¶¡£¼ÓËÙ×îÏÔÖøµÄ»ù×¼²âÊÔÊÇ int4_gemm¡¢ops_gemm ºÍ streamk_gemm¡£
¿ÉÒÔÊӲ쵽£¬¸ßЧµÄÓ²¼þÔÓÀýÈç ldmatrix ºÍ stmatrix£©ÔÚÕâЩºËÖб»¹ã·ºÓÃÓڽṹת»»ÒÔ¼°¹²ÏíÄÚ´æµÄ¼ÓÔØºÍ´æ´¢²Ù×÷¡£ÖµµÃ×¢ÒâµÄÊÇ£¬layer_norm ʵÏÖÁË´Ó 0.99 ±¶µ½ 1.57 ±¶µÄ¼ÓËÙ ¡ª¡ª ÔÚ²î±ðÐÎ×´Ö®¼äÌåÏÖ³öÁËÏÔÖø²î±ð¡£¹ØÓÚijЩÊäÈëÐÎ×´£¬Triton-Linear Äܹ»¼ì²â¡¸µÈЧ¡¹½á¹¹Ö®¼äµÄת»»£¬´Ó¶ø½«×ª»»Àú³Ì½µµÍΪ no-op£¨ÎÞ²Ù×÷£©¡£ÕâÖÖÓÅ»¯Ôھɰæ½á¹¹ÏµÍ³ÖÐÎÞ·¨ÊµÏÖ£¬ÒòΪËüÎÞ·¨Ö±½Ó±È½Ï²î±ðÀàÐ͵Ľṹ£¨ÀýÈ磬Blocked ½á¹¹ºÍ Sliced ½á¹¹£©¡£
ÔÚ RTX4090 ÉÏ£¬ÐÂÒªÁìʵÏÖÁË 1.00 ±¶µ½ 1.51 ±¶µÄ¼ÓËÙ¡£ÓÉÓÚ mma (RTX4090) ºÍ wgmma (GH200) Ö¸ÁîÖ®¼äµÄ²î±ð£¬ËûÃÇÔÚ template_attention ÉÏʵÏÖÁ˸ü¸ßµÄ¼ÓËÙ¡£ÔÚ±¾ÀýÖУ¬tt.dot ÔËËãµÄ×ó²Ù×÷ÊýÔÚÑ»·Íⲿ½ç˵£¬»áÖØ¸´´ÓͬһµØµã¼ÓÔØÊý¾Ý£¬Òò´Ë ldmatrix ºÍͨÀý¹²ÏíÄÚ´æÖ¸Áî¾ù¿ÉʵÏÖ¸ßÍÌÍÂÁ¿¡£ËäÈ»ÓÒ²Ù×÷ÊýÔÚÿ´Îµü´úÖж¼»á¸üУ¬µ« wgmma »áÖ±½ÓÔÚ¹²ÏíÄÚ´æÖлá¼ûËü£¬Ö»ÓÐÔÚ RTX4090 ÉÏ£¬¾¹ýÓÅ»¯ºó£¬Ëü²Å»á±»½µ¼¶µ½ ldmatrix ÖС£Òò´Ë£¬ÔÚ GH200 ÉÏʵÏֵļÓËÙÏà¶Ô½ÏµÍ¡£ÔÚ MI250 ÉÏ£¬ÐÂÒªÁìʵÏÖÁË 0.98 ±¶µ½ 1.18 ±¶µÄ¼ÓËÙ¡£
×ÜÌå¶øÑÔ£¬ÓÉÓÚȱ·¦ ldmatrix µÈ¸ßЧµÄÓ²¼þÔÓTriton-Linear ÔÚ AMD GPU ÉÏʵÏֵļÓËÙµÍÓÚÔÚӢΰ´ï GPU µÄ¡£
¹ØÓÚ OpenAI Open µÄÕâ¸öÑо¿£¬ÄãÓÐʲô¿´·¨ÄØ
ÓÎÏ·ÁÁµã
1¡¢¸»ºñ¶à²ÊµÄÐÞÏÉÍæ·¨
³ýÁËÅàÑøÃÅÉúºÍ½¨ÉèÏÉÃÅÍ⣬ÓÎÏ·»¹°üÀ¨ÁËÁ¶µ¤¡¢Á¶Æ÷¡¢ÏÉÌïµÈ¶àÖÖÐÞÏÉÍæ·¨£¬ÈÃÍæ¼ÒÌåÑéµ½ÐÞÏɵķ½·½ÃæÃæ¡£
2¡¢×ÔÓɽ¨ÉèµÄÏÉÃŵزú
Íæ¼Ò¿ÉÒÔ×ÔÓɰڷÅÐÞÏÉ×ÚÃŵĽ¨Öþ£¬´òÔìÊôÓÚ×Ô¼ºµÄÐÞÏɼÒÔ°£¬´´Á¢ÏÉÃÅÈ˵ÄÀíÏëÊÀ½ç¡£
3¡¢¾«ÃÀϸÄåµÄÓÎÏ·»Ãæ
ÓÎÏ·µÄ»Ã澫ÖÂϸÄ壬ÿһ¸ö³¡¾°¶¼³äÂúÁ˹ŵäÃÀ¸Ð£¬ÈÃÍæ¼ÒËÆºõÉíÁÙÆä¾³£¬¸ÐÊܵ½ÐÞÏÉÖ®ÃÀ¡£
4¡¢Éç½»»¥¶¯µÄÀÖȤ
ÓÎÏ·ÄÚÖø»ºñµÄÉ罻ϵͳ£¬Íæ¼Ò¿ÉÒÔÓëÆäËûÍæ¼Ò×é³ÉÁªÃË£¬ÅäºÏ¶Ô¿¹Ç¿µÐ£¬ÌåÑé¶àÈËÏàÖúµÄÀÖȤ£¬Ôö¼ÓÁËÓÎÏ·µÄ¿ÉÍæÐÔºÍȤζÐÔ¡£
ÓÎÏ·ÆÀ²â
1¡¢ÓÎÏ·Íæ·¨¸»ºñ£¬ÄÚÈÝÉî¶ÈÊ®×㣬¸øÍæ¼Ò´øÀ´Á˼«¼ÑµÄÓÎÏ·ÌåÑé¡£2¡¢»Ã澫ÃÀ£¬³¡¾°Éè¼ÆÎ¨ÃÀ£¬ÈÃÍæ¼ÒÌÕ×íÆäÖУ¬¸ÐÊܵ½ÁËÐÞÏÉÊÀ½çµÄÆæ»ÃÃÀ¸Ð¡£
3¡¢¹Ò»úϵͳµÄÉèÖÃÌùÐÄʵÓ㬽â·ÅÁËÍæ¼ÒµÄË«ÊÖ£¬ÈÃÍæ¼Ò¸üÇáËɵØÏíÊÜÓÎÏ·ÀÖȤ¡£
4¡¢ÃÅÉú¸öÐÔ»¯ËÜÔìÍ»³ö£¬Ã¿¸öÃÅÉú¶¼ÓÐ×Ô¼ºÆæÌصĹÊʺÍÌØµã£¬Ôö¼ÓÁËÓÎÏ·µÄȤζÐԺͿÉÍæÐÔ¡£
¸üÐÂÈÕÖ¾
v8.1.0°æ±¾
1.1µ÷½âÎʶ¦²Ôñ·Ï¯Î»ÈüµÄ·ÀÊØÕóÈÝÉèÖùæÔò£¬ÏÖÔÚÈκÎʱºò¶¼¿ÉÒÔµ÷½â·ÀÊØÕóÈÝ
1.2ÓÅ»¯ÌìµÀÊ÷ÁìÎòµÀ¹ûʱµÀ¹ûÊýÁ¿È±·¦µÄ»ñÈ¡Ìáʾ£¬ÏÖÔÚ»á×Ô¶¯·¿ªµÀ¹û±¦Ï䣬±ãµ±×æÊ¦¿ì½Ý»ñÈ¡
1.3ÓÅ»¯ÐÂÔöÏÉ·¨ÎʵÀͶ×ÊÔ˶¯µÄ¹ºÖÃÌáʾ£¬ÏÖÔÚÐÝÈüÆÚ¹ºÖÃͶ×Êʱ£¬Èç¹ûÎÞ·¨ÄÃÂú½±ÀøÔò»áÓжþ´ÎÈ·ÈÏÌáʾ
1.4ÐÞ¸´Á¬ÐøÁ¶ÖÆÍ¬ÖÖµ¤Ò©Ê±£¬Á¶ÖÆÖÊÁϵÄÊýÁ¿ÏÔʾÒì³£µÄ¹ýʧ
ÏÂÔØµØµã
- µçÄÔ°æ
- /°²×¿°æ
- ÍâµØÏÂÔØÍ¨µÀ£º
- ½öÏÂÔØAPKÎļþ
ͬÀàÓÎÏ·
ÍøÓÑÆÀÂÛ
¹²0ÌõÆÀÂÛ·ÖÀàÁбí
ÀàËÆÓÎÏ·
-
孕妇胸的隐私㊙️图片高清 v8.9.19 ÐÝÏÐÒæÖÇ / 6.5G
-
део+18一29×îаæÏÂÔØ Ä£Äâ¾Óª / 1.21MB
-
❤老师张开腿让我爽了一夜短篇¹Ù·½°æÏÂÔØ ½ÇÉ«ÊÎÑÝ / 7.90G
-
灰原嗯~啊~轻 v6.857.358548 Ä£Äâ¾Óª / 1.62MB
-
Z〇Z○女人另类Z〇Zo挤奶ÍêÕû°æÏÂÔØ ½ÇÉ«ÊÎÑÝ / 5.7G
¾«²Ê·¢Ã÷
»»Ò»»»¾«Æ·ÍƼö
-
www.呦香阁 v8.210.437117 ¾ÓªÑø³É / 636.11MB
¼ì²ì -
男男18禁㊙️啪啪污污内裤 v5.841.420794 ¾ÓªÑø³É / 1.65MB
¼ì²ì -
奢今熙儿20161130 v7.5.4 ¾ÓªÑø³É / 9.43G
¼ì²ì -
教练在泳池下添我做爰 v1.518.461734 ¾ÓªÑø³É / 577.14MB
¼ì²ì -
梦见月瑞希本子18+ v3.324.625435 ¾ÓªÑø³É / 956.28MB
¼ì²ì