Teacher forcing
Teacher forcing（教師監壓）係一種演算法用於訓練遞迴神經網絡（RNN）權重嘅[1]。種演算法涉及反饋啲觀察到嘅序迾值（即ground-truth（做參照嘅真實嘢）樣本）返到RNN喺每一步之後，逼RNN跟實實條ground-truth序迾[2]。
隻詞「teacher forcing」啟發跟攞 RNN 比擬成戥參加一場考試嘅人類學生，場嘢裏頭隻答案畀每隻部分（例如畀計數）取決於前一部分嘅答案。喺種類比裏頭，教師（teacher）嘸係單純做評分畀隻答案喺盡㞘嘅，噉子可能有風險係學生會因為搞錯了開頭噻部份就冚唪唥錯齊孻尾每一橛答案；反而教師會記錄每橛分數畀每隻單獨部分，而且會講隻正確答案畀學生，隻攞來計下一橛嘅。[3]
使到外部teacher訊號即戥實時遞迴學習（real time recurrent learning，RTRL）形成對比[4]。 Teacher訊號係得跟啲振盪器網絡來。[5]隻承諾講係 teacher forcing 有幫減少到訓練時間[6]。
「Teacher forcing」一詞得Ronald J. Williams同埋David Zipser引入喺 1989 年，佢哋報告講隻技術嗰陣時經已「經常得用喺動態監督式學習任務裏頭」。[7][2]
考
編輯- ↑ John F. Kolen; Stefan C. Kremer (15 January 2001). A Field Guide to Dynamical Recurrent Networks. John Wiley & Sons. pp. 202–. ISBN 978-0-7803-5369-5.
- ↑ 2.0 2.1 2.2 Lamb, Alex M; Goyal, Anirudh; Zhang, Ying; Zhang, Saizheng; Courville, Aaron C; Bengio, Yoshua (2016). "Professor Forcing: A New Algorithm for Training Recurrent Networks". Advances in Neural Information Processing Systems. Curran Associates, Inc. 29.
- ↑ Wong, Wanshun (2019-10-15). "What is Teacher Forcing?". Towards Data Science (英文). 喺2022-03-25搵到.
- ↑ Zhang, Ming (31 July 2008). Artificial Higher Order Neural Networks for Economics and Business. IGI Global. pp. 195–. ISBN 978-1-59904-898-7.
- ↑ Yves Chauvin; David E. Rumelhart (1 February 2013). Backpropagation: Theory, Architectures, and Applications. Psychology Press. pp. 473–. ISBN 978-1-134-77581-1.
- ↑ George Bekey; Kenneth Y. Goldberg (30 November 1992). Neural Networks in Robotics. Springer Science & Business Media. pp. 247–. ISBN 978-0-7923-9268-2.
- ↑ Williams, Ronald J.; Zipser, David (June 1989). "A Learning Algorithm for Continually Running Fully Recurrent Neural Networks". Neural Computation. 1 (2): 270–280. doi:10.1162/neco.1989.1.2.270. ISSN 0899-7667.