2012年12月17日

Theano shared variable

http://deeplearning.net/software/theano/tutorial/examples.html#using-shared-variables
裡面的參考範例如下:
from theano import shared
state = shared(0)
inc = T.iscalar('inc')
accumulator = function([inc], state;
                              updates=[(state, state+inc)])

  • 裡面比較特殊的部份是state為shared variable, 0為其初始值。
  • 此值可在多個function當中共用, 在程式當中可用state.get_value()的方式取其值,也可用state.set_value(val)的方式來設定其值。
  • 另一需說明的部份為function.update([shared-variable, new-expression]), 此函數必須為pair form,也可使用dict的key=value形式。
  • 此式的意義即在每次執行時,都將shared-variable.value更換成new-expression所得到的結果。
因此在執行範例後得到的結果如下:
state.get_value() #程式尚未執行,array(0)
accumulator(1)    #array(0)->array(1)
state.get_value() #array(1)
accumulator(300)  #array(1)->array(301)
state.get_value() #array(301)
#reset shared variable
state.set_value(-1)
accumulator(3)
state.get_value() #array(-1)->array(2)
如同上述,shard variable可被多個function共用,因此定義另一個decreaser對state做存取:
decrementor = function([inc], state, updates=[(state, state-inc)])
decrementor(2)
state.get_value() #array(2)->array(0)
如果要在shared variable放函數時,需改用function.given(),範例如下:
fn_of_state = state * 2 + inc
# the type (lscalar) must match the shared ariable we
# are replacing with the ``givens`` list
foo = T.lscalar() 
skip_shared = function([inc, foo], fn_of_state,
                                   givens=[(state, foo)])
skip_shared(1, 3)  # we're using 3 for the state, not state.value
state.get_value()  # old state still there, but we didn't use it
#array(0)
雖然上述的函數相當方便,但文件中未提到是否會有race condition的情形發生。
http://deeplearning.net/software/theano/tutorial/aliasing.html
在understanding memory aliasing for speed and correctness這一節中,提到了theano有自已管理記憶體的機制(pool),而theano會管理pool中變數之變動。

  • theano的pool中的變數與python的變數位於不同的memory space,因此不會互相衝突
  • Theano functions only modify buffers that are in Theano’s memory space.
  • Theano's memory space includes the buffers allocated to store shared variables and the temporaries used to evaluate functions.
  • Physically, Theano's memory space may be spread across the host, a GPU device(s), and in the future may even include objects on a remote machine.
  • The memory allocated for a shared variable buffer is unique: it is never aliased to anothershared variable.
  • Theano's managed memory is constant while Theano functions are not running and Theano's library code is not running.
  • The default behaviour of a function is to return user-space values for outputs, and to expect user-space values for inputs.
The distinction between Theano-managed memory and user-managed memory can be broken down by some Theano functions (e.g. sharedget_value and the constructors for In and Out) by using aborrow=True flag. This can make those methods faster (by avoiding copy operations) at the expense of risking subtle bugs in the overall program (by aliasing memory).