%PDF-1.3
1 0 obj
<<
/Kids [ 4 0 R 5 0 R 6 0 R 7 0 R 8 0 R 9 0 R 10 0 R 11 0 R 12 0 R ]
/Type /Pages
/Count 9
>>
endobj
2 0 obj
<<
/Subject (Neural Information Processing Systems http\072\057\057nips\056cc\057)
/Publisher (Curran Associates)
/Language (en\055US)
/Created (2009)
/Description-Abstract (Although it is widely believed that reinforcement learning is a suitable tool for describing behavioral learning\054 the mechanisms by which it can be implemented in networks of spiking neurons are not fully understood\056 Here\054 we show that different learning rules emerge from a policy gradient approach depending on which features of the spike trains are assumed to influence the reward signals\054 i\056e\056\054 depending on which neural code is in effect\056 We use the framework of Williams \0501992\051 to derive learning rules for arbitrary neural codes\056 For illustration\054 we present policy\055gradient rules for three different example codes \055 a spike count code\054 a spike timing code and the most general \140\140full spike train code \055 and test them on simple model problems\056 In addition to classical synaptic learning\054 we derive learning rules for intrinsic parameters that control the excitability of the neuron\056 The spike count learning rule has structural similarities with established Bienenstock\055Cooper\055Munro rules\056 If the distribution of the relevant spike train features belongs to the natural exponential family\054 the learning rules have a characteristic shape that raises interesting prediction problems\056)
/Producer (Python PDF Library \055 http\072\057\057pybrary\056net\057pyPdf\057)
/Title (Code\055specific policy gradient rules for spiking neurons)
/Date (2009)
/Type (Conference Proceedings)
/firstpage (1741)
/Book (Advances in Neural Information Processing Systems 22)
/Description (Paper accepted and presented at the Neural Information Processing Systems Conference \050http\072\057\057nips\056cc\057\051)
/Editors (Y\056 Bengio and D\056 Schuurmans and J\056D\056 Lafferty and C\056K\056I\056 Williams and A\056 Culotta)
/Author (Henning Sprekeler\054 Guillaume Hennequin\054 Wulfram Gerstner)
/lastpage (1749)
>>
endobj
3 0 obj
<<
/Type /Catalog
/Pages 1 0 R
>>
endobj
4 0 obj
<<
/Contents 13 0 R
/Rotate 0
/Resources <<
/Font <<
/T1_4 14 0 R
/T1_5 18 0 R
/T1_2 23 0 R
/T1_3 27 0 R
/T1_0 31 0 R
/T1_1 35 0 R
>>
/ProcSet [ /PDF /Text ]
>>
/CropBox [ 0 0 612 792 ]
/Parent 1 0 R
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
5 0 obj
<<
/Contents 39 0 R
/Rotate 0
/Resources <<
/Font <<
/T1_6 40 0 R
/T1_7 44 0 R
/T1_4 27 0 R
/T1_5 48 0 R
/T1_2 52 0 R
/T1_3 56 0 R
/T1_0 23 0 R
/T1_1 31 0 R
/T1_8 60 0 R
/T1_9 64 0 R
/T1_10 68 0 R
/T1_11 72 0 R
/T1_12 35 0 R
>>
/ProcSet [ /PDF /Text ]
>>
/CropBox [ 0 0 612 792 ]
/Parent 1 0 R
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
6 0 obj
<<
/Contents 76 0 R
/Rotate 0
/Resources <<
/Font <<
/T1_6 35 0 R
/T1_7 40 0 R
/T1_4 23 0 R
/T1_5 72 0 R
/T1_2 56 0 R
/T1_3 44 0 R
/T1_0 52 0 R
/T1_1 48 0 R
/T1_8 60 0 R
/T1_9 68 0 R
/T1_10 31 0 R
>>
/ProcSet [ /PDF /Text ]
>>
/CropBox [ 0 0 612 792 ]
/Parent 1 0 R
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
7 0 obj
<<
/Contents 77 0 R
/Rotate 0
/Resources <<
/Font <<
/T1_6 60 0 R
/T1_7 72 0 R
/T1_4 48 0 R
/T1_5 44 0 R
/T1_2 56 0 R
/T1_3 68 0 R
/T1_0 23 0 R
/T1_1 52 0 R
/T1_8 35 0 R
/T1_9 31 0 R
/T1_10 27 0 R
/T1_11 78 0 R
>>
/ProcSet [ /PDF /Text ]
>>
/CropBox [ 0 0 612 792 ]
/Parent 1 0 R
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
8 0 obj
<<
/Contents 82 0 R
/Rotate 0
/Resources <<
/ExtGState <<
/GS0 83 0 R
>>
/ColorSpace <<
/CS6 84 0 R
/CS5 88 0 R
/CS4 90 0 R
/CS3 91 0 R
/CS2 93 0 R
/CS1 95 0 R
/CS0 91 0 R
>>
/XObject <<
/Im26 97 0 R
/Im27 98 0 R
/Im24 99 0 R
/Im25 100 0 R
/Im22 101 0 R
/Im23 102 0 R
/Im20 103 0 R
/Im21 104 0 R
/Im28 105 0 R
/Im29 106 0 R
/Im17 107 0 R
/Im30 108 0 R
/Im35 109 0 R
/Im15 110 0 R
/Im14 111 0 R
/Im34 112 0 R
/Im13 113 0 R
/Im12 114 0 R
/Im11 115 0 R
/Im10 116 0 R
/Im31 117 0 R
/Im16 118 0 R
/Im33 119 0 R
/Im32 120 0 R
/Im19 121 0 R
/Im18 122 0 R
/Im7 123 0 R
/Im6 124 0 R
/Im5 125 0 R
/Im4 126 0 R
/Im3 127 0 R
/Im2 128 0 R
/Im1 129 0 R
/Im0 130 0 R
/Im9 131 0 R
/Im8 132 0 R
>>
/Font <<
/T1_6 133 0 R
/T1_7 48 0 R
/T1_4 137 0 R
/T1_5 141 0 R
/T1_2 145 0 R
/T1_3 149 0 R
/T1_0 23 0 R
/T1_1 31 0 R
/T1_8 52 0 R
/T1_9 60 0 R
/T1_10 44 0 R
/T1_11 72 0 R
/T1_12 56 0 R
/T1_13 35 0 R
/T1_14 68 0 R
>>
/ProcSet [ /PDF /Text /ImageC /ImageI ]
>>
/CropBox [ 0 0 612 792 ]
/Parent 1 0 R
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
9 0 obj
<<
/Contents 153 0 R
/Rotate 0
/Resources <<
/Font <<
/T1_6 72 0 R
/T1_7 35 0 R
/T1_4 44 0 R
/T1_5 60 0 R
/T1_2 56 0 R
/T1_3 48 0 R
/T1_0 23 0 R
/T1_1 52 0 R
/T1_8 68 0 R
/T1_9 31 0 R
/T1_10 78 0 R
>>
/ProcSet [ /PDF /Text ]
>>
/CropBox [ 0 0 612 792 ]
/Parent 1 0 R
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
10 0 obj
<<
/Contents 154 0 R
/Rotate 0
/Resources <<
/Font <<
/T1_6 48 0 R
/T1_7 60 0 R
/T1_4 31 0 R
/T1_5 56 0 R
/T1_2 52 0 R
/T1_3 72 0 R
/T1_0 44 0 R
/T1_1 23 0 R
>>
/ProcSet [ /PDF /Text ]
>>
/CropBox [ 0 0 612 792 ]
/Parent 1 0 R
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
11 0 obj
<<
/Contents 155 0 R
/Rotate 0
/Resources <<
/Font <<
/T1_6 31 0 R
/T1_7 27 0 R
/T1_4 56 0 R
/T1_5 35 0 R
/T1_2 48 0 R
/T1_3 44 0 R
/T1_0 23 0 R
/T1_1 52 0 R
>>
/ProcSet [ /PDF /Text ]
>>
/CropBox [ 0 0 612 792 ]
/Parent 1 0 R
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
12 0 obj
<<
/Contents 156 0 R
/Rotate 0
/Resources <<
/Font <<
/T1_0 23 0 R
/T1_1 27 0 R
>>
/ProcSet [ /PDF /Text ]
>>
/CropBox [ 0 0 612 792 ]
/Parent 1 0 R
/MediaBox [ 0 0 612 792 ]
/Type /Page
>>
endobj
13 0 obj
<<
/Length 2923
/Filter /FlateDecode
>>
stream
HWY+~XվH4M)b>-6s$WKN_of>(Xp}>]_b_qDj."zwsqco(Lw.2?-]~}ck}n2/WQx7ۘ.KjQvA}z]eaQ*y5E;=l7oϏo?_&ċvL]eer?]Z~\"X~݇-&6P_ #A#;qxBdFMq$[L?Ԝ{
wnsheH"H(JHK:^wV9CƶiTjtm
F4H %~Aˢ,B?"?#memsuuɁϳF
Gvf AWOj]jrIϪ h d/ 'Џi{U6/J\.J"r.U3^tqyfoAo]'M>LIL9^(^&tˑE <_%ɌD!hm#`_qZUoNe]>
!R}NAimasՙtTWY Un/ ~M: @Y`|+Α?Y,(Q0FA(Ԑ 6u5E79"xNTڱZě6z3rKE/[Sx瑋Թ` 8q$O`[*7Ջ X.9l7!qNs^Ѳx1L5es#^U`.H/9Jx[xw] 'NiDv%ߣue)Wsa9jM(F,@N;%zcxO{٨vXoeASrNo֠s@KP@i<kzWqrd.4Q
M&t}Q$.auukyQ"&6(
eyp֯n~t ܔ-}yiEgX;KcziaEwZK{"yfx!5!Y;[Ԛ %fA}U]2yę ީۈ1`lx6i\0Ӎ=%~S#]yEE4fZd};rm-M&u+ѱqr%) KFЯvof
D4u[ZF<%z=|c
UV2kS Lx{{
̉LΦKؐ.SN@oR9G;_Xd%S,6xYQ|U5h碅Σ(k3!x07Wus+4(3C \N6Rs}ԔAo~Mׁ1 FI~,ܡEY$KQ0pn9w=XT[IqSfXJN±pfQQ
e}IY7DRT"9!Z{K,(so%i`A8