8jtddlmZddlmZmZddlmZddlmZm Z ddl m Z ddl m Z mZmZmZddlmZmZmZddlmZmZdd lmZdd lmZmZer"dd lmZmZdd l m!Z!dd l"m#Z#ddl$m%Z%ddl&m'Z'm(Z(e deGddZ)e dGddeZ*e dGddeZ+y)) annotations)asynccontextmanagercontextmanager) dataclass) TYPE_CHECKINGAny) docs_group)DatasetDatasetResponseDatasetStatisticsDatasetStatisticsResponse)DEFAULT_CHUNK_SIZEget_items_iteratorget_items_iterator_async)ResourceClientResourceClientAsync) create_storage_content_signature)response_to_dictresponse_to_list) AsyncIteratorIterator) timedelta) GeneralAccess) HttpResponse)JsonSerializableTimeoutOthercXeZdZUdZded< ded< ded< ded< ded< d ed <y ) DatasetItemsPagea A page of dataset items returned by the `list_items` method. Dataset items are arbitrary JSON objects stored in the dataset, so they cannot be represented by a specific Pydantic model. This class provides pagination metadata along with the raw items. zlist[dict[str, Any]]itemsinttotaloffsetcountlimitbooldescN)__name__ __module__ __qualname____doc____annotations__]/var/www/html/GAP/venv/lib/python3.12/site-packages/apify_client/_resource_clients/dataset.pyrrs: I J/ K4 Jt J3 J;r.rzResource clientsceZdZdZddd dfdZddddZdddd  dd Zdddd Zddddddddddddd d ddZdddddddddddd d d dZ dddddddddddddddddd d d!dZ e ddddddddddddddddd d d"dZ ddd#dZ ddd$dZddddddddddddd d d%dZxZS)& DatasetClientzSub-client for managing a specific dataset. Provides methods to manage a specific dataset, e.g. get it, update it, or download its items. Obtain an instance via an appropriate method on the `ApifyClient` class. Ndatasets resource_id resource_pathc *t|d||d|yNr3r-super__init__selfr4r5kwargs __class__s r/r:zDatasetClient.__init__<'  #'  r.shorttimeoutcj|j|}|ytj|jSaRetrieve the dataset. https://docs.apify.com/api/v2#/reference/datasets/dataset/get-dataset Args: timeout: Timeout for the API HTTP request. Returns: The retrieved dataset, or None, if it does not exist. rAN_getr model_validatedatar<rBresults r/getzDatasetClient.getIs37+ >--f5:::r.namegeneral_accessrBch|j|||}tj|jS)wUpdate the dataset with specified fields. https://docs.apify.com/api/v2#/reference/datasets/dataset/update-dataset Args: name: The new name for the dataset. general_access: Determines how others can access the dataset. timeout: Timeout for the API HTTP request. Returns: The updated dataset. rBrM generalAccess_updater rGrHr<rMrNrBrJs r/updatezDatasetClient.updateYs9&(  --f5:::r.c(|j|yzDelete the dataset. https://docs.apify.com/api/v2#/reference/datasets/dataset/delete-dataset Args: timeout: Timeout for the API HTTP request. rAN_deleter<rBs r/deletezDatasetClient.deletess W %r.long r#r%cleanr'fieldsomitunwind skip_empty skip_hiddenflattenview signaturerBc |j||||||||| | | |  }|jj|jdd|| }t |}t |t |jdt |jdtt |jdt|t |jd|jd jd k( S) 3 List the items of the dataset. https://docs.apify.com/api/v2#/reference/datasets/item-collection/get-items Args: offset: Number of items that should be skipped at the start. The default value is 0. limit: Maximum number of items to return. By default there is no limit. desc: By default, results are returned in the same order as they were stored. To reverse the order, set this parameter to True. clean: If True, returns only non-empty items and skips hidden fields (i.e. fields starting with the # character). The clean parameter is just a shortcut for skip_hidden=True and skip_empty=True parameters. Note that since some objects might be skipped from the output, that the result might contain less items than the limit value. fields: A list of fields which should be picked from the items, only these fields will remain in the resulting record objects. Note that the fields in the outputted items are sorted the same way as they are specified in the fields parameter. You can use this feature to effectively fix the output format. omit: A list of fields which should be omitted from the items. unwind: A list of fields which should be unwound, in order which they should be processed. Each field should be either an array or an object. If the field is an array then every element of the array will become a separate record and merged with parent object. If the unwound field is an object then it is merged with the parent object. If the unwound field is missing or its value is neither an array nor an object and therefore cannot be merged with a parent object, then the item gets preserved as it is. Note that the unwound items ignore the desc parameter. skip_empty: If True, then empty items are skipped from the output. Note that if used, the results might contain less items than the limit value. skip_hidden: If True, then hidden fields are skipped from the output, i.e. fields starting with the # character. flatten: A list of fields that should be flattened. view: Name of the dataset view to be used. signature: Signature used to access the items. timeout: Timeout for the API HTTP request. Returns: A page of the list of dataset items according to the specified filters. r#r%r'r_r`rarb skipEmpty skipHiddenrerfrgr GETurlmethodparamsrBx-apify-pagination-totalx-apify-pagination-offsetx-apify-pagination-countx-apify-pagination-limitx-apify-pagination-desctruer r"r#r$r%r' _build_params _http_clientcall _build_urlrrr!headersmaxlenlowerr<r#r%r_r'r`rarbrcrdrerfrgrBrequest_paramsresponser s r/ list_itemszDatasetClient.list_items}sj++ ",  $$))(! * !*h&&'ABCx''(CDEc(**+EFGUTh&&'ABC!!";<BBDN  r. r#r%r_r'r`rarbrcrdrg chunk_sizerBc j dddd  f d} t| ||| xstS)` Iterate over the items in the dataset. Simple `list_items` does only one API call, possibly not listing all items matching the criteria. This method returns an iterator that is capable of making multiple API calls to retrieve all items matching the criteria. https://docs.apify.com/api/v2#/reference/datasets/item-collection/get-items Args: offset: Number of items that should be skipped at the start. The default value is 0. limit: Maximum number of dataset rows to scan. Fewer items are yielded when filters drop some, more when `unwind` splits a row into several. By default there is no limit. desc: By default, results are returned in the same order as they were stored. To reverse the order, set this parameter to True. clean: If True, returns only non-empty items and skips hidden fields (i.e. fields starting with the # character). The clean parameter is just a shortcut for skip_hidden=True and skip_empty=True parameters. Note that since some objects might be skipped from the output, that the result might contain less items than the limit value. fields: A list of fields which should be picked from the items, only these fields will remain in the resulting record objects. Note that the fields in the outputted items are sorted the same way as they are specified in the fields parameter. You can use this feature to effectively fix the output format. omit: A list of fields which should be omitted from the items. unwind: A list of fields which should be unwound, in order which they should be processed. Each field should be either an array or an object. If the field is an array then every element of the array will become a separate record and merged with parent object. If the unwound field is an object then it is merged with the parent object. If the unwound field is missing or its value is neither an array nor an object and therefore cannot be merged with a parent object, then the item gets preserved as it is. Note that the unwound items ignore the desc parameter. skip_empty: If True, then empty items are skipped from the output. Note that if used, the results might contain less items than the limit value. skip_hidden: If True, then hidden fields are skipped from the output, i.e. fields starting with the # character. signature: Signature used to access the items. chunk_size: Maximum number of dataset rows requested per API call when iterating across pages. timeout: Timeout for the API HTTP request. Yields: An item from the dataset. Nr%r#c < j||    SN) r#r%r_r'r`rarbrcrdrgrBr r%r#r_r'r`rar<rgrcrdrBrbs r/ _callbackz.DatasetClient.iterate_items.._callbacks:??%'##  r.r%r#rr% int | Noner#rreturnr)rrr<r#r%r_r'r`rarbrcrdrgrrBrs` ```````` ` r/ iterate_itemszDatasetClient.iterate_itemss5p.2   ")5T^Ttbtuur.json item_formatr#r%r'r_bom delimiterr`rarbrcskip_header_rowrdxml_rootxml_rowrergrBc|j||||||||| | | | | ||||}|jj|jdd||}|jS)Get the items in the dataset as raw bytes. https://docs.apify.com/api/v2#/reference/datasets/item-collection/get-items Args: item_format: Format of the results, possible values are: json, jsonl, csv, html, xlsx, xml and rss. The default value is json. offset: Number of items that should be skipped at the start. The default value is 0. limit: Maximum number of items to return. By default there is no limit. desc: By default, results are returned in the same order as they were stored. To reverse the order, set this parameter to True. clean: If True, returns only non-empty items and skips hidden fields (i.e. fields starting with the # character). The clean parameter is just a shortcut for skip_hidden=True and skip_empty=True parameters. Note that since some objects might be skipped from the output, that the result might contain less items than the limit value. bom: All text responses are encoded in UTF-8 encoding. By default, csv files are prefixed with the UTF-8 Byte Order Mark (BOM), while json, jsonl, xml, html and rss files are not. If you want to override this default behavior, specify bom=True query parameter to include the BOM or bom=False to skip it. delimiter: A delimiter character for CSV files. The default delimiter is a simple comma (,). fields: A list of fields which should be picked from the items, only these fields will remain in the resulting record objects. Note that the fields in the outputted items are sorted the same way as they are specified in the fields parameter. You can use this feature to effectively fix the output format. You can use this feature to effectively fix the output format. omit: A list of fields which should be omitted from the items. unwind: A list of fields which should be unwound, in order which they should be processed. Each field should be either an array or an object. If the field is an array then every element of the array will become a separate record and merged with parent object. If the unwound field is an object then it is merged with the parent object. If the unwound field is missing or its value is neither an array nor an object and therefore cannot be merged with a parent object, then the item gets preserved as it is. Note that the unwound items ignore the desc parameter. skip_empty: If True, then empty items are skipped from the output. Note that if used, the results might contain less items than the limit value. skip_header_row: If True, then header row in the csv format is skipped. skip_hidden: If True, then hidden fields are skipped from the output, i.e. fields starting with the # character. xml_root: Overrides default root element name of xml output. By default the root element is items. xml_row: Overrides default element name that wraps each page or page function result object in xml output. By default the element name is item. flatten: A list of fields that should be flattened. signature: Signature used to access the items. timeout: Timeout for the API HTTP request. Returns: The dataset items as raw bytes. formatr#r%r'r_rrr`rarbrk skipHeaderRowrlxmlRootxmlRowrergr rmrnrzr{r|r}contentr<rr#r%r'r_rrr`rarbrcrrdrrrergrBrrs r/get_items_as_bytesz DatasetClient.get_items_as_bytes sJ++ )"#, ($$))(! * r.rr#r%r'r_rrr`rarbrcrrdrrrgrBc#Kd} |j||||||||| | | | | |||}|jj|jdd|d|}||r|j yy#|r|j wwxYwwa Retrieve the items in the dataset as a stream. https://docs.apify.com/api/v2#/reference/datasets/item-collection/get-items Args: item_format: Format of the results, possible values are: json, jsonl, csv, html, xlsx, xml and rss. The default value is json. offset: Number of items that should be skipped at the start. The default value is 0. limit: Maximum number of items to return. By default there is no limit. desc: By default, results are returned in the same order as they were stored. To reverse the order, set this parameter to True. clean: If True, returns only non-empty items and skips hidden fields (i.e. fields starting with the # character). The clean parameter is just a shortcut for skip_hidden=True and skip_empty=True parameters. Note that since some objects might be skipped from the output, that the result might contain less items than the limit value. bom: All text responses are encoded in UTF-8 encoding. By default, csv files are prefixed with the UTF-8 Byte Order Mark (BOM), while json, jsonl, xml, html and rss files are not. If you want to override this default behavior, specify bom=True query parameter to include the BOM or bom=False to skip it. delimiter: A delimiter character for CSV files. The default delimiter is a simple comma (,). fields: A list of fields which should be picked from the items, only these fields will remain in the resulting record objects. Note that the fields in the outputted items are sorted the same way as they are specified in the fields parameter. You can use this feature to effectively fix the output format. You can use this feature to effectively fix the output format. omit: A list of fields which should be omitted from the items. unwind: A list of fields which should be unwound, in order which they should be processed. Each field should be either an array or an object. If the field is an array then every element of the array will become a separate record and merged with parent object. If the unwound field is an object then it is merged with the parent object. If the unwound field is missing or its value is neither an array nor an object and therefore cannot be merged with a parent object, then the item gets preserved as it is. Note that the unwound items ignore the desc parameter. skip_empty: If True, then empty items are skipped from the output. Note that if used, the results might contain less items than the limit value. skip_header_row: If True, then header row in the csv format is skipped. skip_hidden: If True, then hidden fields are skipped from the output, i.e. fields starting with the # character. xml_root: Overrides default root element name of xml output. By default the root element is items. xml_row: Overrides default element name that wraps each page or page function result object in xml output. By default the element name is item. signature: Signature used to access the items. timeout: Timeout for the API HTTP request. Returns: The dataset items as a context-managed streaming `Response`. N)rr#r%r'r_rrr`rarbrkrrlrrrgr rmT)rorprqstreamrB)rzr{r|r}closer<rr#r%r'r_rrr`rarbrcrrdrrrgrBrrs r/ stream_itemszDatasetClient.stream_itemssH !!//"#$-& #!0N&((--OOG,% .HN x sBAA.B.BBmediumc d}d}t|tr|}n|}|jj|j ddddi|j |||yaXPush items to the dataset. https://docs.apify.com/api/v2#/reference/datasets/item-collection/put-items Args: items: The items which to push in the dataset. Either a stringified JSON, a dictionary, or a list of strings or dictionaries. timeout: Timeout for the API HTTP request. Nr POSTz content-typezapplication/json; charset=utf-8)rorpr~rqrHrrB isinstancestrr{r|r}rzr<r rBrHrs r/ push_itemszDatasetClient.push_itemssh eS !DD (#%FG%%'  r.c|jj|jdd|j|}t |}t j |jS)JGet the dataset statistics. https://docs.apify.com/api/v2#tag/DatasetsStatistics/operation/dataset_statistics_get Args: timeout: Timeout for the API HTTP request. Returns: The dataset statistics. Raises: NotFoundError: If the dataset does not exist. statisticsrmrnr{r|r}rzrr rGrHr<rBrrJs r/get_statisticszDatasetClient.get_statisticss`$$)) -%%' *  "(+(77?DDDr. r#r%r_r'r`rarbrcrdrerf expires_inrBc |j| }|j||||||||| | |  }|r3|jr't|j|j| }||d<|j d|S)Generate a URL that can be used to access dataset items. If the client has permission to access the dataset's URL signing key, the URL will include a signature to verify its authenticity. You can optionally control how long the signed URL should be valid using the `expires_in` option. This value sets the expiration duration from the time the URL is generated. If not provided, the URL will not expire. Any other options (like `limit` or `offset`) will be included as query parameters in the URL. Args: offset: Number of items that should be skipped at the start. The default value is 0. limit: Maximum number of items to return. By default there is no limit. clean: If True, returns only non-empty items and skips hidden fields. desc: By default, results are returned in the same order as they were stored. To reverse the order, set this parameter to True. fields: A list of fields which should be picked from the items. omit: A list of fields which should be omitted from the items. unwind: A list of fields which should be unwound, in order which they should be processed. skip_empty: If True, then empty items are skipped from the output. skip_hidden: If True, then hidden fields are skipped from the output. flatten: A list of fields that should be flattened. view: Name of the dataset view to be used. expires_in: How long the signed URL should be valid. timeout: Timeout for the API HTTP request. Returns: The public dataset items URL. rA r#r%r'r_r`rarbrkrlrerfrrgr rKrzurl_signing_secret_keyrid_build_public_urlr<r#r%r_r'r`rarbrcrdrerfrrBdatasetrrgs r/create_items_public_urlz%DatasetClient.create_items_public_urls^((7(+++ ",   w558 ..%I +4N; '%%g~>>r.r4 str | Noner5rr=rrNonerBrrzDataset | NonerMrrNzGeneralAccess | NonerBrrr rBrrrr#rr%rr_ bool | Noner'rr`list[str] | NonerarrbrrcrrdrrerrfrrgrrBrrr)r#rr%rr_rr'rr`rrarrbrrcrrdrrgrrrrBrrzIterator[dict]&rrr#rr%rr'rr_rrrrrr`rrarrbrrcrrrrdrrrrrrerrgrrBrrbytes)$rrr#rr%rr'rr_rrrrrr`rrarrbrrcrrrrdrrrrrrgrrBrrzIterator[HttpResponse]r rrBrrrrBrrr r#rr%rr_rr'rr`rrarrbrrcrrdrrerrfrrztimedelta | NonerBrrr)r(r)r*r+r:rKrVr\rrrrrrrr __classcell__r>s@r/r1r14s#''           )0;& /3" ;;- ;  ;  ;4,3&" ! #'!%#'"&#'$( $!X X  X  X  X !X X !X  X !X "X X X X !X z" ! #'!%#'"&#' $!%!GvGv Gv  Gv  Gv!GvGv!Gv Gv!GvGvGvGv GvX"!  ! $#'!%#'"&'+#'#"$( $!)` `  `  `  ` ` ` ` !` ` !`  ` %` !` !` "#` $"%` &'` ()` * +` D"!  ! $#'!%#'"&'+#'#" $!'b!b! b!  b!  b!b!b!b!!b!b!!b! b!%b!!b! !b!"#b!$%b!&'b!( )b!b!HIQ 84;E4" ! #'!%#'"&#'$('+!G?G? G?  G?  G?!G?G?!G? G?!G?"G?G?%G?G? !G?r.r1ceZdZdZddd dfdZddddZdddd  dd Zdddd Zddddddddddddd d ddZdddddddddddd d d dZ dddddddddddddddddd d d!dZ e ddddddddddddddddd d d"dZ ddd#dZ ddd$dZddddddddddddd d d%dZxZS)&DatasetClientAsynczSub-client for managing a specific dataset. Provides methods to manage a specific dataset, e.g. get it, update it, or download its items. Obtain an instance via an appropriate method on the `ApifyClientAsync` class. Nr2r3c *t|d||d|yr7r8r;s r/r:zDatasetClientAsync.__init__lr?r.r@rAcK|j|d{}|ytj|jS7&wrDrErIs r/rKzDatasetClientAsync.getys?yyy11 >--f5:::2sA?'ArLcK|j|||d{}tj|jS7#w)rPrQNrSrUs r/rVzDatasetClientAsync.updatesG&||($   --f5:::  sA>$AcDK|j|d{y7wrXrYr[s r/r\zDatasetClientAsync.deletesll7l+++s   r]r^c K|j||||||||| | | |  }|jj|jdd|| d{}t |}t |t |jdt |jdtt |jdt|t |jd |jd jd k( S7w) rirjr rmrnNrrrsrtrurvrwrxryrs r/rzDatasetClientAsync.list_itemssj++ ",  **//(! 0  !*h&&'ABCx''(CDEc(**+EFGUTh&&'ABC!!";<BBDN   sAC=C;B*C=rc j dddd  f d} t| ||| xstS)rNrc X Kj||    d{S7wrrrs r/rz3DatasetClientAsync.iterate_items.._callback?sH%'#)    s *(*rr)rrrs` ```````` ` r/rz DatasetClientAsync.iterate_itemss9p48d   ( U6j>^L^  r.rrcK|j||||||||| | | | | ||||}|jj|jdd||d{}|jS7w)rrr rmrnNrrs r/rz%DatasetClientAsync.get_items_as_bytesRsJ++ )"#, (**//(! 0   sAA)A'A)rcLKd} |j||||||||| | | | | |||}|jj|jdd|d|d{}||r|j d{yy7%7#|r|j d{7wwxYwwr)rzr{r|r}aclosers r/rzDatasetClientAsync.stream_itemssH (!//"#$-& #!0N&"..33OOG,% 4HNoo'''(oo'''sMB$ABA? B#B$8B9B$?BB$B!BB!!B$rc Kd}d}t|tr|}n|}|jj|j ddddi|j |||d{y7wrrrs r/rzDatasetClientAsync.push_itemssv eS !DD$$(#%FG%%'%   sAA( A&!A(cK|jj|jdd|j|d{}t |}t j |jS7.w)rrrmrnNrrs r/rz!DatasetClientAsync.get_statistics5sn**// -%%' 0   "(+(77?DDD sAA3A1/A3rc K|j| d{}|j||||||||| | |  }|r3|jr't|j|j| }||d<|j d|S7gw)rrANrrrgr rrs r/rz*DatasetClientAsync.create_items_public_urlLs^11++ ",   w558 ..%I +4N; '%%g~>>12sBBA(Brrrrr)r#rr%rr_rr'rr`rrarrbrrcrrdrrgrrrrBrrzAsyncIterator[dict]r)$rrr#rr%rr'rr_rrrrrr`rrarrbrrcrrrrdrrrrrrgrrBrrzAsyncIterator[HttpResponse]rrr)r(r)r*r+r:rKrVr\rrrrrrrrrrs@r/rrds#''           /6;& /3" ;;- ;  ;  ;429," ! #'!%#'"&#'$( $!X X  X  X  X !X X !X  X !X "X X X X !X z" ! #'!%#'"&#' $!%!I I  I  I  I !I I !I  I !I I I I  I \"!  ! $#'!%#'"&'+#'#"$( $!)` `  `  `  ` ` ` ` !` ` !`  ` %` !` !` "#` $"%` &'` ()` * +` D"!  ! $#'!%#'"&'+#'#" $!'b(b( b(  b(  b(b(b(b(!b(b(!b( b(%b(!b( !b("#b($%b(&'b(( %)b(b(HOW 8:AE4" ! #'!%#'"&#'$('+!G?G? G?  G?  G?!G?G?!G? G?!G?"G?G?%G?G? !G?r.rN), __future__r contextlibrr dataclassesrtypingrrapify_client._docsr apify_client._modelsr r r r apify_client._paginationrrr/apify_client._resource_clients._resource_clientrrapify_client._utils.cryptorapify_client._utils.httprrcollections.abcrrdatetimerapify_client._literalsrapify_client.http_clientsrapify_client.typesrrrr1rr-r.r/rs":!%)ggee_GG7"46< G << <6 l?Nl? l?^ n?,n? n?r.