cdocutils.nodes document q)q}q(U nametypesq}q(X/an introduction to boto's dynamodb v2 interfaceqNXusing an existing tableqNXqueryingqNXdeleting an itemq NXdeleting a tableq NXupdating an itemq NX batch writingq NX parallel scanq NXdynamodbqX next stepsqNX getting an item & accessing dataqNXiterator protocolqXcreating a new itemqNX batch readingqNXamazon dynamodb localqX dynamodb2_tutqXcreating a new tableqNXdynamodb localqNXthe high-level apiqNX the resultsetqNuUsubstitution_defsq}qUparse_messagesq]qUcurrent_sourceqNU decorationqNUautofootnote_startq KUnameidsq!}q"(hU/an-introduction-to-boto-s-dynamodb-v2-interfaceq#hUusing-an-existing-tableq$hUqueryingq%h Udeleting-an-itemq&h Udeleting-a-tableq'h Uupdating-an-itemq(h U batch-writingq)h U parallel-scanq*hUdynamodbq+hU next-stepsq,hUgetting-an-item-accessing-dataq-hUiterator-protocolq.hUcreating-a-new-itemq/hU batch-readingq0hUamazon-dynamodb-localq1hU dynamodb2-tutq2hUcreating-a-new-tableq3hUdynamodb-localq4hUthe-high-level-apiq5hU the-resultsetq6uUchildrenq7]q8(cdocutils.nodes target q9)q:}q;(U rawsourceqXC/Users/kyleknap/Documents/GitHub/boto/docs/source/dynamodb2_tut.rstq?Utagnameq@UtargetqAU attributesqB}qC(UidsqD]UbackrefsqE]UdupnamesqF]UclassesqG]UnamesqH]UrefidqIh2uUlineqJKUdocumentqKhh7]ubcdocutils.nodes section qL)qM}qN(hh?Uexpect_referenced_by_nameqO}qPhh:sh@UsectionqQhB}qR(hF]hG]hE]hD]qS(h#h2ehH]qT(hheuhJKhKhUexpect_referenced_by_idqU}qVh2h:sh7]qW(cdocutils.nodes title qX)qY}qZ(hh?h@Utitleq\hB}q](hF]hG]hE]hD]hH]uhJKhKhh7]q^cdocutils.nodes Text q_X/An Introduction to boto's DynamoDB v2 interfaceq`qa}qb(hh?h@U paragraphqfhB}qg(hF]hG]hE]hD]hH]uhJKhKhh7]qh(h_X4This tutorial focuses on the boto interface to AWS' qiqj}qk(hh?h@hAhB}q~(hrhshD]qh+ahE]hF]hG]hH]qhauhJK hKhh7]ubcdocutils.nodes warning q)q}q(h`.h=hMh>h?h@UwarningqhB}q(hF]hG]hE]hD]hH]uhJNhKhh7]qhc)q}q(h`.h=hh>h?h@hfhB}q(hF]hG]hE]hD]hH]uhJKh7]q(h_XThis tutorial covers the qq}q(h`qh=hh>h?h@U pending_xrefqhB}q(UreftypeXdocqUrefwarnqU reftargetqX dynamodb_tutU refdomainUhD]hE]U refexplicithF]hG]hH]UrefdocqX dynamodb2_tutquhJKh7]qcdocutils.nodes literal q)q}q(hh?h@hfhB}q(hF]hG]hE]hD]hH]uhJKhKhh7]q(h_XoThe v2 DynamoDB API has both a high-level & low-level component. The low-level API (contained primarily within qq}q(hh?h@hfhB}q(hF]hG]hE]hD]hH]uhJKhKhh7]qh_X~The high-level API attempts to make interacting with the service more natural from Python. It supports most of the featureset.qɅq}q(hh?h@hQhB}q(hF]hG]hE]hD]qh5ahH]qhauhJKhKhh7]q(hX)q}q(hh?h@h\hB}q(hF]hG]hE]hD]hH]uhJKhKhh7]qh_XThe High-Level APIqׅq}q(hh?h@hfhB}q(hF]hG]hE]hD]hH]uhJKhKhh7]q(h_X<Most of the interaction centers around a single object, the qޅq}q(hh?h@hQhB}q(hF]hG]hE]hD]qh3ahH]qhauhJK%hKhh7]q(hX)q}q(hh?h@h\hB}q(hF]hG]hE]hD]hH]uhJK%hKhh7]qh_XCreating a New Tableqq}q(hh?h@hfhB}q(hF]hG]hE]hD]hH]uhJK'hKhh7]q(h_X(To create a new table, you need to call qq}q(hh?h@hfhB}r (hF]hG]hE]hD]hH]uhJK*hKhh7]rh_X/Since both the key schema and local secondary indexes can not be modified after the table is created, you'll need to plan ahead of time how you think the table will be used. Both the keys & indexes are also used for querying, so you'll want to represent the data you'll need when querying there as well.rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJK0hKhh7]r(h_X-For the schema, you can either have a single rr}r(hr?}r@(hh?h@hfhB}rW(hF]hG]hE]hD]hH]uhJK8hKhh7]rX(h_X8For the local secondary indexes, you can choose from an rYrZ}r[(h duplicates only the keys from the schema onto the index. The rr}r(h duplicates only the keys from the schema onto the index. The h=jUubh)r}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJK?hKhh7]rh_XSimple example:rr}r(h>> from boto.dynamodb2.fields import HashKey >>> from boto.dynamodb2.table import Table # Uses your ``aws_access_key_id`` & ``aws_secret_access_key`` from either a # config file or environment variable & the default region. >>> users = Table.create('users', schema=[ ... HashKey('username'), ... ])h=hh>h?h@U literal_blockrhB}r(U xml:spacerUpreserverhD]hE]hF]hG]hH]uhJKAhKhh7]rh_X0>>> from boto.dynamodb2.fields import HashKey >>> from boto.dynamodb2.table import Table # Uses your ``aws_access_key_id`` & ``aws_secret_access_key`` from either a # config file or environment variable & the default region. >>> users = Table.create('users', schema=[ ... HashKey('username'), ... ])rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJKJhKhh7]rh_XA full example:rr}r(h>> import boto.dynamodb2 >>> from boto.dynamodb2.fields import HashKey, RangeKey, KeysOnlyIndex, AllIndex >>> from boto.dynamodb2.table import Table >>> from boto.dynamodb2.types import NUMBER >>> users = Table.create('users', schema=[ ... HashKey('account_type', data_type=NUMBER), ... RangeKey('last_name'), ... ], throughput={ ... 'read': 5, ... 'write': 15, ... }, indexes=[ ... AllIndex('EverythingIndex', parts=[ ... HashKey('account_type', data_type=NUMBER), ... ]) ... ], ... # If you need to specify custom parameters like keys or region info... ... connection= boto.dynamodb2.connect_to_region('us-east-1'))h=hh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJKLhKhh7]rh_X>>> import boto.dynamodb2 >>> from boto.dynamodb2.fields import HashKey, RangeKey, KeysOnlyIndex, AllIndex >>> from boto.dynamodb2.table import Table >>> from boto.dynamodb2.types import NUMBER >>> users = Table.create('users', schema=[ ... HashKey('account_type', data_type=NUMBER), ... RangeKey('last_name'), ... ], throughput={ ... 'read': 5, ... 'write': 15, ... }, indexes=[ ... AllIndex('EverythingIndex', parts=[ ... HashKey('account_type', data_type=NUMBER), ... ]) ... ], ... # If you need to specify custom parameters like keys or region info... ... connection= boto.dynamodb2.connect_to_region('us-east-1'))rr}r(hh?h@hQhB}r(hF]hG]hE]hD]rh$ahH]rhauhJKahKhh7]r(hX)r}r(hh?h@h\hB}r(hF]hG]hE]hD]hH]uhJKahKhh7]rh_XUsing an Existing Tablerr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJKchKhh7]r(h_X^Once a table has been created, using it is relatively simple. You can either specify just the rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJKhhKhh7]rh_X Lazy example:rr}r(h>> from boto.dynamodb2.table import Table >>> users = Table('users')h=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJKjhKhh7]rh_XE>>> from boto.dynamodb2.table import Table >>> users = Table('users')rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJKmhKhh7]rh_XEfficient example:rr}r(h>> from boto.dynamodb2.fields import HashKey, RangeKey, AllIndex >>> from boto.dynamodb2.table import Table >>> from boto.dynamodb2.types import NUMBER >>> users = Table('users', schema=[ ... HashKey('account_type', data_type=NUMBER), ... RangeKey('last_name'), ... ], indexes=[ ... AllIndex('EverythingIndex', parts=[ ... HashKey('account_type', data_type=NUMBER), ... ]) ... ])h=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJKohKhh7]rh_X>>> from boto.dynamodb2.fields import HashKey, RangeKey, AllIndex >>> from boto.dynamodb2.table import Table >>> from boto.dynamodb2.types import NUMBER >>> users = Table('users', schema=[ ... HashKey('account_type', data_type=NUMBER), ... RangeKey('last_name'), ... ], indexes=[ ... AllIndex('EverythingIndex', parts=[ ... HashKey('account_type', data_type=NUMBER), ... ]) ... ])rr }r (hh?h@hQhB}r (hF]hG]hE]hD]rh/ahH]rhauhJK}hKhh7]r(hX)r}r(hh?h@h\hB}r(hF]hG]hE]hD]hH]uhJK}hKhh7]rh_XCreating a New Itemrr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJKhKhh7]r(h_XOnce you have a rr}r(hh?h@hfhB}r,(hF]hG]hE]hD]hH]uhJKhKhh7]r-(h_XThe first is to use the r.r/}r0(hh_Xmustr?r@}rA(hh?h@hfhB}rQ(hF]hG]hE]hD]hH]uhJKhKhh7]rRh_XExample:rSrT}rU(h>> from boto.dynamodb2.table import Table >>> users = Table('users') # Create the new user. >>> users.put_item(data={ ... 'username': 'johndoe', ... 'first_name': 'John', ... 'last_name': 'Doe', ... }) Trueh=j h>h?h@jhB}rX(jjhD]hE]hF]hG]hH]uhJKhKhh7]rYh_X>>> from boto.dynamodb2.table import Table >>> users = Table('users') # Create the new user. >>> users.put_item(data={ ... 'username': 'johndoe', ... 'first_name': 'John', ... 'last_name': 'Doe', ... }) TruerZr[}r\(hh?h@hfhB}r_(hF]hG]hE]hD]hH]uhJKhKhh7]r`(h_X,The alternative is to manually construct an rarb}rc(hh?h@hfhB}r{(hF]hG]hE]hD]hH]uhJKhKhh7]r|h_XExample:r}r~}r(h>> from boto.dynamodb2.items import Item >>> from boto.dynamodb2.table import Table >>> users = Table('users') # WARNING - This doens't save it yet! >>> johndoe = Item(users, data={ ... 'username': 'johndoe', ... 'first_name': 'John', ... 'last_name': 'Doe', ... }) # The data now gets persisted to the server. >>> johndoe.save() Trueh=j h>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJKhKhh7]rh_X\>>> from boto.dynamodb2.items import Item >>> from boto.dynamodb2.table import Table >>> users = Table('users') # WARNING - This doens't save it yet! >>> johndoe = Item(users, data={ ... 'username': 'johndoe', ... 'first_name': 'John', ... 'last_name': 'Doe', ... }) # The data now gets persisted to the server. >>> johndoe.save() Truerr}r(hh?h@hQhB}r(hF]hG]hE]hD]rh-ahH]rhauhJKhKhh7]r(hX)r}r(hh?h@h\hB}r(hF]hG]hE]hD]hH]uhJKhKhh7]rh_X Getting an Item & Accessing Datarr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJKhKhh7]r(h_X}With data now in DynamoDB, if you know the key of the item, you can fetch it back out. Specify the key value(s) as kwargs to rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJKhKhh7]rh_XExample:rr}r(h>> from boto.dynamodb2.table import Table >>> users = Table('users') >>> johndoe = users.get_item(username='johndoe')h=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJKhKhh7]rh_Xw>>> from boto.dynamodb2.table import Table >>> users = Table('users') >>> johndoe = users.get_item(username='johndoe')rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJKhKhh7]r(h_XOnce you have an rr}r(h>> johndoe = users.get_item(username='johndoe') # Read a field out. >>> johndoe['first_name'] 'John' # Change a field (DOESN'T SAVE YET!). >>> johndoe['first_name'] = 'Johann' # Delete data from it (DOESN'T SAVE YET!). >>> del johndoe['last_name']h=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJKhKhh7]rh_X>>> johndoe = users.get_item(username='johndoe') # Read a field out. >>> johndoe['first_name'] 'John' # Change a field (DOESN'T SAVE YET!). >>> johndoe['first_name'] = 'Johann' # Delete data from it (DOESN'T SAVE YET!). >>> del johndoe['last_name']rr}r(hh?h@hQhB}r(hF]hG]hE]hD]rh(ahH]rh auhJKhKhh7]r(hX)r}r(hh?h@h\hB}r(hF]hG]hE]hD]hH]uhJKhKhh7]rh_XUpdating an Itemrr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJKhKhh7]r(h_XFJust creating new items or changing only the in-memory version of the rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJKhKhh7]rh_XThe first is sending all the data with the expectation nothing has changed since you read the data. DynamoDB will verify the data is in the original state and, if so, will send all of the item's data. If that expectation fails, the call will fail:rr}r(h>> johndoe = users.get_item(username='johndoe') >>> johndoe['first_name'] = 'Johann' >>> johndoe['whatever'] = "man, that's just like your opinion" >>> del johndoe['last_name'] # Affects all fields, even the ones not changed locally. >>> johndoe.save() Trueh=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJKhKhh7]rh_X>>> johndoe = users.get_item(username='johndoe') >>> johndoe['first_name'] = 'Johann' >>> johndoe['whatever'] = "man, that's just like your opinion" >>> del johndoe['last_name'] # Affects all fields, even the ones not changed locally. >>> johndoe.save() Truerr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJKhKhh7]rh_XThe second is a full overwrite. If you can be confident your version of the data is the most correct, you can force an overwrite of the data.:rr}r(h>> johndoe = users.get_item(username='johndoe') >>> johndoe['first_name'] = 'Johann' >>> johndoe['whatever'] = "man, that's just like your opinion" >>> del johndoe['last_name'] # Specify ``overwrite=True`` to fully replace the data. >>> johndoe.save(overwrite=True) Trueh=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJKhKhh7]rh_X>>> johndoe = users.get_item(username='johndoe') >>> johndoe['first_name'] = 'Johann' >>> johndoe['whatever'] = "man, that's just like your opinion" >>> del johndoe['last_name'] # Specify ``overwrite=True`` to fully replace the data. >>> johndoe.save(overwrite=True) Truerr}r(hh?h@hfhB}r (hF]hG]hE]hD]hH]uhJKhKhh7]r h_XThe last is a partial update. If you've only modified certain fields, you can send a partial update that only writes those fields, allowing other (potentially changed) fields to go untouched.:r r }r (h>> johndoe = users.get_item(username='johndoe') >>> johndoe['first_name'] = 'Johann' >>> johndoe['whatever'] = "man, that's just like your opinion" >>> del johndoe['last_name'] # Partial update, only sending/affecting the # ``first_name/whatever/last_name`` fields. >>> johndoe.partial_save() Trueh=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJKhKhh7]rh_X+>>> johndoe = users.get_item(username='johndoe') >>> johndoe['first_name'] = 'Johann' >>> johndoe['whatever'] = "man, that's just like your opinion" >>> del johndoe['last_name'] # Partial update, only sending/affecting the # ``first_name/whatever/last_name`` fields. >>> johndoe.partial_save() Truerr}r(hh?h@hQhB}r(hF]hG]hE]hD]rh&ahH]rh auhJKhKhh7]r(hX)r}r(hh?h@h\hB}r(hF]hG]hE]hD]hH]uhJKhKhh7]rh_XDeleting an Itemr r!}r"(hh?h@hfhB}r&(hF]hG]hE]hD]hH]uhJKhKhh7]r'h_XhYou can also delete items from the table. You have two choices, depending on what data you have present.r(r)}r*(hh?h@hfhB}r-(hF]hG]hE]hD]hH]uhJKhKhh7]r.(h_XIf you already have an r/r0}r1(h(hF]hG]hE]hD]hH]uh=j+h7]r?h_X Item.deleter@rA}rB(h>> johndoe.delete() Trueh=jh>h?h@jhB}rH(jjhD]hE]hF]hG]hH]uhJKhKhh7]rIh_X>>> johndoe.delete() TruerJrK}rL(hh?h@hfhB}rO(hF]hG]hE]hD]hH]uhJMhKhh7]rP(h_XIf you don't have an rQrR}rS(h>> from boto.dynamodb2.table import Table >>> users = Table('users') >>> users.delete_item(username='johndoe') Trueh=jh>h?h@jhB}rt(jjhD]hE]hF]hG]hH]uhJMhKhh7]ruh_Xu>>> from boto.dynamodb2.table import Table >>> users = Table('users') >>> users.delete_item(username='johndoe') Truervrw}rx(hh?h@hQhB}r{(hF]hG]hE]hD]r|h)ahH]r}h auhJM hKhh7]r~(hX)r}r(hh?h@h\hB}r(hF]hG]hE]hD]hH]uhJM hKhh7]rh_X Batch Writingrr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMhKhh7]rh_XIf you're loading a lot of data at a time, making use of batch writing can both speed up the process & reduce the number of write requests made to the service.rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMhKhh7]r(h_XrBatch writing involves wrapping the calls you want batched in a context manager. The context manager imitates the rr}r(h>> from boto.dynamodb2.table import Table >>> users = Table('users') >>> with users.batch_write() as batch: ... batch.put_item(data={ ... 'username': 'anotherdoe', ... 'first_name': 'Another', ... 'last_name': 'Doe', ... 'date_joined': int(time.time()), ... }) ... batch.put_item(data={ ... 'username': 'alice', ... 'first_name': 'Alice', ... 'date_joined': int(time.time()), ... }) ... batch.delete_item(username=jane')h=jyh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJMhKhh7]rh_X>>> from boto.dynamodb2.table import Table >>> users = Table('users') >>> with users.batch_write() as batch: ... batch.put_item(data={ ... 'username': 'anotherdoe', ... 'first_name': 'Another', ... 'last_name': 'Doe', ... 'date_joined': int(time.time()), ... }) ... batch.put_item(data={ ... 'username': 'alice', ... 'first_name': 'Alice', ... 'date_joined': int(time.time()), ... }) ... batch.delete_item(username=jane')rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJM(hKhh7]rh_XRHowever, there are some limitations on what you can do within the context manager.rr}r(hh?h@U bullet_listrhB}r(UbulletrX*hD]hE]hF]hG]hH]uhJM+hKhh7]r(cdocutils.nodes list_item r)r}r(hh?h@U list_itemrhB}r(hF]hG]hE]hD]hH]uhJNhKhh7]rhc)r}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJM+h7]rh_X;It can't read data at all or do batch any other operations.rr}r(hh?h@jhB}r(hF]hG]hE]hD]hH]uhJNhKhh7]rhc)r}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJM,h7]rh_X<You can't put & delete the same data within a batch request.rr}r(hh?h@UnoterhB}r(hF]hG]hE]hD]hH]uhJNhKhh7]rhc)r}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJM0h7]r(h_XAdditionally, the context manager can only batch 25 items at a time for a request (this is a DynamoDB limitation). It is handled for you so you can keep writing additional items, but you should be aware that 100 rr}r(hh?h@hQhB}r(hF]hG]hE]hD]rh%ahH]rhauhJM7hKhh7]r(hX)r}r(hh?h@h\hB}r(hF]hG]hE]hD]hH]uhJM7hKhh7]rh_XQueryingrr}r(hh?h@hhB}r(hF]hG]hE]hD]hH]uhJNhKhh7]rhc)r}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJM;h7]r(h_XThe rr}r (hh=jh>h?h@hfhB}r?(hF]hG]hE]hD]hH]uhJM@hKhh7]r@h_XManually fetching out each item by itself isn't tenable for large datasets. To cope with fetching many records, you can either perform a standard query, query via a local secondary index or scan the entire table.rArB}rC(hh=j<ubaubhc)rD}rE(hh?h@hfhB}rF(hF]hG]hE]hD]hH]uhJMDhKhh7]rG(h_XyA standard query typically gets run against a hash+range key combination. Filter parameters are passed as kwargs & use a rHrI}rJ(hh?h@hfhB}rW(hF]hG]hE]hD]hH]uhJMHhKhh7]rXh_XIn terms of querying, our original schema is less than optimal. For the following examples, we'll be using the following table setup:rYrZ}r[(h>> users = Table.create('users', schema=[ ... HashKey('account_type'), ... RangeKey('last_name'), ... ], indexes=[ ... AllIndex('DateJoinedIndex', parts=[ ... HashKey('account_type'), ... RangeKey('date_joined', data_type=NUMBER), ... ]), ... ])h=jh>h?h@jhB}r^(jjhD]hE]hF]hG]hH]uhJMKhKhh7]r_h_X>>> users = Table.create('users', schema=[ ... HashKey('account_type'), ... RangeKey('last_name'), ... ], indexes=[ ... AllIndex('DateJoinedIndex', parts=[ ... HashKey('account_type'), ... RangeKey('date_joined', data_type=NUMBER), ... ]), ... ])r`ra}rb(hh?h@hfhB}rf(hF]hG]hE]hD]hH]uhJMUhKhh7]rgh_XWhen executing the query, you get an iterable back that contains your results. These results may be spread over multiple requests as DynamoDB paginates them. This is done transparently, but you should be aware it may take more than one request.rhri}rj(hh?h@hfhB}rn(hF]hG]hE]hD]hH]uhJMZhKhh7]roh_X;To run a query for last names starting with the letter "D":rprq}rr(h>> names_with_d = users.query_2( ... account_type__eq='standard_user', ... last_name__beginswith='D' ... ) >>> for user in names_with_d: ... print user['first_name'] 'Bob' 'Jane' 'John'h=jh>h?h@jhB}ru(jjhD]hE]hF]hG]hH]uhJM\hKhh7]rvh_X>>> names_with_d = users.query_2( ... account_type__eq='standard_user', ... last_name__beginswith='D' ... ) >>> for user in names_with_d: ... print user['first_name'] 'Bob' 'Jane' 'John'rwrx}ry(hh?h@hfhB}r|(hF]hG]hE]hD]hH]uhJMghKhh7]r}(h_XYou can also reverse results (r~r}r(h>> rev_with_d = users.query_2( ... account_type__eq='standard_user', ... last_name__beginswith='D', ... reverse=True, ... limit=2 ... ) >>> for user in rev_with_d: ... print user['first_name'] 'John' 'Jane'h=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJMjhKhh7]rh_X>>> rev_with_d = users.query_2( ... account_type__eq='standard_user', ... last_name__beginswith='D', ... reverse=True, ... limit=2 ... ) >>> for user in rev_with_d: ... print user['first_name'] 'John' 'Jane'rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMvhKhh7]r(h_X]You can also run queries against the local secondary indexes. Simply provide the index name (rr}r(h>> recent = users.query_2( ... account_type__eq='standard_user', ... date_joined__gte=time.time() - (60 * 60), ... index='DateJoinedIndex' ... ) >>> for user in recent: ... print user['first_name'] 'Alice' 'Jane'h=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJMzhKhh7]rh_X# Users within the last hour. >>> recent = users.query_2( ... account_type__eq='standard_user', ... date_joined__gte=time.time() - (60 * 60), ... index='DateJoinedIndex' ... ) >>> for user in recent: ... print user['first_name'] 'Alice' 'Jane'rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMhKhh7]r(h_XBy default, DynamoDB can return a large amount of data per-request (up to 1Mb of data). To prevent these requests from drowning other smaller gets, you can specify a smaller page size via the rr}r(h>> all_users = users.query_2( ... account_type__eq='standard_user', ... date_joined__gte=0, ... max_page_size=10 ... ) # Usage is the same, but now many smaller requests are done. >>> for user in recent: ... print user['first_name'] 'Alice' 'Jane'h=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJMhKhh7]rh_X]# Small pages yield faster responses & less potential of drowning other # requests. >>> all_users = users.query_2( ... account_type__eq='standard_user', ... date_joined__gte=0, ... max_page_size=10 ... ) # Usage is the same, but now many smaller requests are done. >>> for user in recent: ... print user['first_name'] 'Alice' 'Jane'rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMhKhh7]r(h_X_Finally, if you need to query on data that's not in either a key or in an index, you can run a rr}r(hh?h@hhB}r(hF]hG]hE]hD]hH]uhJNhKhh7]rhc)r}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMh7]rh_XScans are consistent & run over the entire table, so relatively speaking, they're more expensive than plain queries or queries against an LSI.rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMhKhh7]rh_X7An example scan of all records in the table looks like:rr}r(h>> all_users = users.scan()h=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJMhKhh7]rh_X>>> all_users = users.scan()r r }r (hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMhKhh7]rh_XFiltering a scan looks like:rr}r(h>> owners_with_emails = users.scan( ... is_owner__eq=1, ... email__null=False, ... ) >>> for user in recent: ... print user['first_name'] 'George' 'John'h=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJMhKhh7]rh_X>>> owners_with_emails = users.scan( ... is_owner__eq=1, ... email__null=False, ... ) >>> for user in recent: ... print user['first_name'] 'George' 'John'rr}r(hh?h@hQhB}r(hF]hG]hE]hD]rh6ahH]rhauhJMhKhh7]r (hX)r!}r"(hh?h@h\hB}r$(hF]hG]hE]hD]hH]uhJMhKhh7]r%(h_XThe r&r'}r((hh?h@hfhB}r4(hF]hG]hE]hD]hH]uhJMhKhh7]r5(h_XBoth r6r7}r8(h}r?(hh?h@hfhB}re(hF]hG]hE]hD]hH]uhJMhKhh7]rf(h_X!Typical use is simply a standard rgrh}ri(h>> result_set = users.scan() >>> for user in result_set: ... print user['first_name']h=jh>h?h@jhB}rv(jjhD]hE]hF]hG]hH]uhJMhKhh7]rwh_XZ>>> result_set = users.scan() >>> for user in result_set: ... print user['first_name']rxry}rz(hh?h@hfhB}r}(hF]hG]hE]hD]hH]uhJMhKhh7]r~(h_XbHowever, this throws away results as it fetches more data. As a result, you can't index it like a rr}r(h>> len(result_set) 0h=jh>Nh@U doctest_blockrhB}r(jjhD]hE]hF]hG]hH]uhJNhKhh7]rh_X>>> len(result_set) 0rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMhKhh7]r(h_XBecause it does this, if you need to loop over your results more than once (or do things like negative indexing, length checks, etc.), you should wrap it in a call to rr}r(h>> result_set = users.scan() >>> all_users = list(result_set) # Slice it for every other user. >>> for user in all_users[::2]: ... print user['first_name']h=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJMhKhh7]rh_X>>> result_set = users.scan() >>> all_users = list(result_set) # Slice it for every other user. >>> for user in all_users[::2]: ... print user['first_name']rr}r(hh?h@hhB}r(hF]hG]hE]hD]hH]uhJNhKhh7]r(hc)r}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMh7]r(h_X!Wrapping calls like the above in rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMh7]r(h_XAppropriate use of the rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMh7]r(h_X2Alternatively, you can build your own list, using rr}r (hh?h@hAhB}r (hrjZhD]r!h.ahE]hF]hG]hH]r"hauhJMhKhh7]ubeubeubhL)r#}r$(hh?h@hQhB}r%(hF]hG]hE]hD]r&h*ahH]r'h auhJMhKhh7]r((hX)r)}r*(hh?h@h\hB}r,(hF]hG]hE]hD]hH]uhJMhKhh7]r-h_X Parallel Scanr.r/}r0(hh?h@hfhB}r3(hF]hG]hE]hD]hH]uhJMhKhh7]r4(h_XYDynamoDB also includes a feature called "Parallel Scan", which allows you to make use of r5r6}r7(h(hh?h@hfhB}rE(hF]hG]hE]hD]hH]uhJMhKhh7]rFh_XThis does require extra code on the user's part & you should ensure that you need the speed boost, have enough data to justify it and have the extra capacity to read it without impacting other queries/scans.rGrH}rI(hTo run it, you should pick the ``total_segments`` to use, which is an integer representing the number of temporary partitions you'd divide your table into. You then need to spin up a thread/process for each one, giving each thread/process a ``segment``, which is a zero-based integer of the segment you'd like to scan.h=j#h>h?h@hfhB}rL(hF]hG]hE]hD]hH]uhJMhKhh7]rM(h_XTo run it, you should pick the rNrO}rP(hh?h@hfhB}rg(hF]hG]hE]hD]hH]uhJMhKhh7]rhh_X[An example of using parallel scan to send out email to all users might look something like:rirj}rk(hh?h@jhB}rn(jjhD]hE]hF]hG]hH]uhJMhKhh7]roh_X#!/usr/bin/env python import threading import boto.ses import boto.dynamodb2 from boto.dynamodb2.table import Table AWS_ACCESS_KEY_ID = '' AWS_SECRET_ACCESS_KEY = '' APPROVED_EMAIL = 'some@address.com' def send_email(email): # Using Amazon's Simple Email Service, send an email to a given # email address. You must already have an email you've verified with # AWS before this will work. conn = boto.ses.connect_to_region( 'us-east-1', aws_access_key_id=AWS_ACCESS_KEY_ID, aws_secret_access_key=AWS_SECRET_ACCESS_KEY ) conn.send_email( APPROVED_EMAIL, "[OurSite] New feature alert!", "We've got some exciting news! We added a new feature to...", [email] ) def process_segment(segment=0, total_segments=10): # This method/function is executed in each thread, each getting its # own segment to process through. conn = boto.dynamodb2.connect_to_region( 'us-east-1', aws_access_key_id=AWS_ACCESS_KEY_ID, aws_secret_access_key=AWS_SECRET_ACCESS_KEY ) table = Table('users', connection=conn) # We pass in the segment & total_segments to scan here. for user in table.scan(segment=segment, total_segments=total_segments): send_email(user['email']) def send_all_emails(): pool = [] # We're choosing to divide the table in 3, then... pool_size = 3 # ...spinning up a thread for each segment. for i in range(pool_size): worker = threading.Thread( target=process_segment, kwargs={ 'segment': i, 'total_segments': pool_size, } ) pool.append(worker) # We start them to let them start scanning & consuming their # assigned segment. worker.start() # Finally, we wait for each to finish. for thread in pool: thread.join() if __name__ == '__main__': send_all_emails()rprq}rr(hh?h@hQhB}ru(hF]hG]hE]hD]rvh0ahH]rwhauhJM=hKhh7]rx(hX)ry}rz(hh?h@h\hB}r|(hF]hG]hE]hD]hH]uhJM=hKhh7]r}h_X Batch Readingr~r}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJM?hKhh7]r(h_XSimilar to batch writing, batch reading can also help reduce the number of API requests necessary to access a large number of items. The rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMDhKhh7]rh_XThis is done lazily, so if you never iterate over the results, no requests are executed. Additionally, if you only iterate over part of the set, the minumum number of calls are made to fetch those results (typically max 100 per response).rr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMIhKhh7]rh_XExample:rr}r(h>> from boto.dynamodb2.table import Table >>> users = Table('users') # No request yet. >>> many_users = users.batch_get(keys=[ {'username': 'alice'}, {'username': 'bob'}, {'username': 'fred'}, {'username': 'jane'}, {'username': 'johndoe'}, ]) # Now the request is performed, requesting all five in one request. >>> for user in many_users: ... print user['first_name'] 'Alice' 'Bobby' 'Fred' 'Jane' 'John'h=jsh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJMKhKhh7]rh_X>>> from boto.dynamodb2.table import Table >>> users = Table('users') # No request yet. >>> many_users = users.batch_get(keys=[ {'username': 'alice'}, {'username': 'bob'}, {'username': 'fred'}, {'username': 'jane'}, {'username': 'johndoe'}, ]) # Now the request is performed, requesting all five in one request. >>> for user in many_users: ... print user['first_name'] 'Alice' 'Bobby' 'Fred' 'Jane' 'John'rr}r(hh?h@hQhB}r(hF]hG]hE]hD]rh'ahH]rh auhJMbhKhh7]r(hX)r}r(hh?h@h\hB}r(hF]hG]hE]hD]hH]uhJMbhKhh7]rh_XDeleting a Tablerr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMdhKhh7]rh_XSDeleting a table is a simple exercise. When you no longer need a table, simply run:rr}r(h>> users.delete()h=jh>h?h@jhB}r(jjhD]hE]hF]hG]hH]uhJMghKhh7]rh_X>>> users.delete()rr}r(hh?h@hQhB}r(hF]hG]hE]hD]rh4ahH]rhauhJMkhKhh7]r(hX)r}r(hh?h@h\hB}r(hF]hG]hE]hD]hH]uhJMkhKhh7]rh_XDynamoDB Localrr}r(hh?h@hfhB}r(hF]hG]hE]hD]hH]uhJMmhKhh7]r(hl)r}r(hh?h@jhB}r(jjhD]hE]hF]hG]hH]uhJMphKhh7]rh_X#!/usr/bin/env python from boto.dynamodb2.layer1 import DynamoDBConnection # Connect to DynamoDB Local conn = DynamoDBConnection( host='localhost', port=8000, aws_secret_access_key='anything', is_secure=False) # List all local tables tables = conn.list_tables()rr}r(hh?h@hAhB}r(hrjhD]rh1ahE]hF]hG]hH]rhauhJMhKhh7]ubeubhL)r}r(hh?h@hQhB}r(hF]hG]hE]hD]rh,ahH]rhauhJMhKhh7]r(hX)r}r(hh?h@h\hB}r(hF]hG]hE]hD]hH]uhJMhKhh7]rh_X Next Stepsrr}r(h`.h=jh>h?h@hfhB}r(hF]hG]hE]hD]hH]uhJMhKhh7]r(h_XQYou can find additional information about other calls & parameter options in the rr}r(h`rh=jh>h?h@hhB}r(UreftypeXdocrhhX ref/dynamodb2U refdomainUhD]hE]U refexplicithF]hG]hH]hhuhJMh7]rh)r }r (h}r?(Ufootnote_backlinksr@KUrecord_dependenciesrANU rfc_base_urlrBUhttp://tools.ietf.org/html/rCU tracebackrDUpep_referencesrENUstrip_commentsrFNU toc_backlinksrGUentryrHU language_coderIUenrJU datestamprKNU report_levelrLKU _destinationrMNU halt_levelrNKU strip_classesrONh\NUerror_encoding_error_handlerrPUbackslashreplacerQUdebugrRNUembed_stylesheetrSUoutput_encoding_error_handlerrTUstrictrUU sectnum_xformrVKUdump_transformsrWNU docinfo_xformrXKUwarning_streamrYNUpep_file_url_templaterZUpep-%04dr[Uexit_status_levelr\KUconfigr]NUstrict_visitorr^NUcloak_email_addressesr_Utrim_footnote_reference_spacer`UenvraNUdump_pseudo_xmlrbNUexpose_internalsrcNUsectsubtitle_xformrdU source_linkreNUrfc_referencesrfNUoutput_encodingrgUutf-8rhU source_urlriNUinput_encodingrjU utf-8-sigrkU_disable_configrlNU id_prefixrmUU tab_widthrnKUerror_encodingroUUTF-8rpU_sourcerqh?Ugettext_compactrrU generatorrsNUdump_internalsrtNU smart_quotesruU pep_base_urlrvUhttp://www.python.org/dev/peps/rwUsyntax_highlightrxUlongryUinput_encoding_error_handlerrzjUUauto_id_prefixr{Uidr|Udoctitle_xformr}Ustrip_elements_with_classesr~NU _config_filesr]Ufile_insertion_enabledrU raw_enabledrKU dump_settingsrNubUsymbol_footnote_startrKUidsr}r(h+h{h5hh-jh3hh(jh'jh)jyh*j#h1jh#hMh/j h4jh.jh&jh%jh$jh,jh0jsh6jh2hMuUsubstitution_namesr}rh@hKhB}r(hF]hD]hE]Usourceh?hG]hH]uU footnotesr]rUrefidsr}rh2]rh:asub.