Error: Error in x[[1]] : subscript out of bounds #62

AdamJayJonca · 2020-08-04T13:24:20Z

When getting the request by date:

search_analytics(`my domain`,
                 "2020-07-29", 
                 Sys.Date()-4, 
                 dimensions =  c(
                     "date",
                     "country",  
                     "query"
                 ),
                  walk_data = "byDate"
)

I get the error:
Error in x[[1]] : subscript out of bounds Calls: sourceWithProgress ... search_analytics -> gar_batch_walk -> Reduce -> lapply -> FUN
The output when the walk_data is byBatch is proper.

options(googleAuthR.verbose=2):

Fetching search analytics for url: xxxxx dates: 2020-07-29 2020-08-01 dimensions: date country query dimensionFilterExp:  searchType: web aggregationType: auto
Batching data via method: byDate
Will fetch up to 25000 rows per day
i 2020-08-04 15:20:01 > Batch API limited to [ 1 ] calls at once.
i 2020-08-04 15:20:01 > Request #:  2020-07-29
i 2020-08-04 15:20:01 > Constructing batch request URL for:  /webmasters/v3/sites/xxxxx/searchAnalytics/query
i 2020-08-04 15:20:02 > Making new batched cache
i 2020-08-04 15:20:02 > Making Batch API call
i 2020-08-04 15:20:04 > Request #:  2020-07-30
i 2020-08-04 15:20:04 > Constructing batch request URL for:  /webmasters/v3/sites/xxxxx/searchAnalytics/query
i 2020-08-04 15:20:04 > Making new batched cache
i 2020-08-04 15:20:04 > Making Batch API call
i 2020-08-04 15:20:07 > Request #:  2020-07-31
i 2020-08-04 15:20:07 > Constructing batch request URL for:  /webmasters/v3/sites/xxxxx/searchAnalytics/query
i 2020-08-04 15:20:07 > Making new batched cache
i 2020-08-04 15:20:07 > Making Batch API call
i 2020-08-04 15:20:09 > Request #:  2020-08-01
i 2020-08-04 15:20:09 > Constructing batch request URL for:  /webmasters/v3/sites/xxxx/searchAnalytics/query
i 2020-08-04 15:20:09 > Making new batched cache
i 2020-08-04 15:20:09 > Making Batch API call
Error in value[[3L]](cond) : 
  Error with batch response - writing response to C:\Users\A7D10~1\AppData\Local\Temp\RtmpeixfLX\filef584e4a786e.rds

Session Info

`R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

Random number generation:
RNG: Mersenne-Twister
Normal: Inversion
Sample: Rounding

locale:
[1] LC_COLLATE=Polish_Poland.1250 LC_CTYPE=Polish_Poland.1250 LC_MONETARY=Polish_Poland.1250 LC_NUMERIC=C LC_TIME=Polish_Poland.1250

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] forcats_0.5.0 stringr_1.4.0 dplyr_1.0.0 purrr_0.3.4 readr_1.3.1 tidyr_1.1.0 tibble_3.0.3 ggplot2_3.3.2 tidyverse_1.3.0
[10] searchConsoleR_0.4.0 googleAnalyticsR_0.8.0 googleAuthR_1.3.0 RJDBC_0.2-8 rJava_0.9-11 DBI_1.1.0

loaded via a namespace (and not attached):
[1] tidyselect_1.1.0 slam_0.1-47 NLP_0.2-0 haven_2.2.0 gargle_0.4.0 lattice_0.20-38 colorspace_1.4-1 vctrs_0.3.1 generics_0.0.2 yaml_2.2.1
[11] utf8_1.1.4 rlang_0.4.7 pillar_1.4.6 glue_1.4.1 withr_2.1.2 dbplyr_1.4.2 readxl_1.3.1 modelr_0.1.6 lifecycle_0.2.0 plyr_1.8.6
[21] cellranger_1.1.0 munsell_0.5.0 gtable_0.3.0 rvest_0.3.5 memoise_1.1.0 tm_0.7-7 parallel_3.6.3 curl_4.3 fansi_0.4.1 broom_0.5.5
[31] Rcpp_1.0.5 openssl_1.4.2 scales_1.1.0 backports_1.1.5 jsonlite_1.7.0 fs_1.3.1 rCharts_0.4.5 hms_0.5.3 askpass_1.1 digest_0.6.25
[41] stringi_1.4.6 RJSONIO_1.3-1.4 grid_3.6.3 cli_2.0.2 tools_3.6.3 magrittr_1.5 crayon_1.3.4 whisker_0.4 pkgconfig_2.0.3 ellipsis_0.3.1
[51] xml2_1.3.2 reprex_0.3.0 lubridate_1.7.4 assertthat_0.2.1.9000 httr_1.4.1 rstudioapi_0.11 R6_2.4.1 nlme_3.1-144 compiler_3.6.3
`

The text was updated successfully, but these errors were encountered:

MarkEdmondson1234 · 2020-08-04T14:39:49Z

I guess that day had no data, but could you load the contents of the file it writes (C:\Users\A7D10~1\AppData\Local\Temp\RtmpeixfLX\filef584e4a786e.rds in your example) using readRDS(file) and give the contents of that R object here? It should confirm the response structure.

AdamJayJonca · 2020-08-05T06:19:08Z

I guess that day had no data, but could you load the contents of the file it writes (C:\Users\A7D10~1\AppData\Local\Temp\RtmpeixfLX\filef584e4a786e.rds in your example) using readRDS(file) and give the contents of that R object here? It should confirm the response structure.

The date range have some data (even for sucha a specific request).
'Try this API' tool response shows the proper data:

curl --request POST \
  'https://www.googleapis.com/webmasters/v3/sites/xxxxxx/searchAnalytics/query?key=[YOUR_API_KEY]' \
  --header 'Authorization: Bearer [YOUR_ACCESS_TOKEN]' \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{"startDate":"2020-07-29","endDate":"2020-08-01","dimensions":["country","query","date"]}' \
  --compressed

HTTP/1.1 200 
cache-control: private
content-encoding: gzip
content-length: 10616
content-type: application/json; charset=UTF-8
date: Wed, 05 Aug 2020 06:13:29 GMT
server: ESF
vary: Origin, X-Origin, Referer

{
  "rows": [
    {
      "keys": [
        "aut",
        "1sfa611620r1001",
        "2020-07-30"
      ],
      "clicks": 1,
      "impressions": 2,
      "ctr": 0.5,
      "position": 4
    },

MarkEdmondson1234 · 2020-08-05T06:41:02Z

Can you load the rds file as it’s a batched call and the response needs to be examined.

…

________________________________ From: AdamJayJonca <[email protected]> Sent: Wednesday, August 5, 2020 8:19:21 AM To: MarkEdmondson1234/searchConsoleR <[email protected]> Cc: Mark <[email protected]>; Comment <[email protected]> Subject: Re: [MarkEdmondson1234/searchConsoleR] Error: Error in x[[1]] : subscript out of bounds (#62) I guess that day had no data, but could you load the contents of the file it writes (C:\Users\A7D10~1\AppData\Local\Temp\RtmpeixfLX\filef584e4a786e.rds in your example) using readRDS(file) and give the contents of that R object here? It should confirm the response structure. The date range have some data (even for sucha a specific request). 'Try this API' tool response shoe the proper data: curl --request POST \ 'https://www.googleapis.com/webmasters/v3/sites/xxxxxx/searchAnalytics/query?key=[YOUR_API_KEY]' \ --header 'Authorization: Bearer [YOUR_ACCESS_TOKEN]' \ --header 'Accept: application/json' \ --header 'Content-Type: application/json' \ --data '{"startDate":"2020-07-29","endDate":"2020-08-01","dimensions":["country","query","date"]}' \ --compressed HTTP/1.1 200 cache-control: private content-encoding: gzip content-length: 10616 content-type: application/json; charset=UTF-8 date: Wed, 05 Aug 2020 06:13:29 GMT server: ESF vary: Origin, X-Origin, Referer { "rows": [ { "keys": [ "aut", "1sfa611620r1001", "2020-07-30" ], "clicks": 1, "impressions": 2, "ctr": 0.5, "position": 4 }, — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#62 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAYCPLAKEIUVPEJHQMC5N23R7D2WTANCNFSM4PUMYJLA>.

AdamJayJonca · 2020-08-05T07:12:02Z

Can you load the rds file as it’s a batched call and the response needs to be examined.

file.zip

MarkEdmondson1234 · 2020-08-05T13:18:48Z

This may be related to #43 but I can replicate it

MarkEdmondson1234 · 2020-08-05T15:09:54Z

This is a subtle problem back in googleAuthR and compounded by the search console API which seems to have variable output, sometimes with leading blank spaces and not which makes it tricky to debug. For now I'd say avoid byDate until it can be sorted out, I'll work on it on this branch https://github.com/MarkEdmondson1234/googleAuthR/tree/batch_indexing

mmueller-cm · 2020-10-16T20:44:41Z

Hey @MarkEdmondson1234 - Fantastic package! Is there any update to the byDate batching? I'm seeing the same issue on my end. It appears to query the requested dates and then fails out

ℹ 2020-10-16 15:38:20 > Request #: 2020-10-06 ℹ 2020-10-16 15:38:22 > Request #: 2020-10-07 ℹ 2020-10-16 15:38:24 > Request #: 2020-10-08 ℹ 2020-10-16 15:38:25 > Request #: 2020-10-09 ℹ 2020-10-16 15:38:27 > Request #: 2020-10-10 ℹ 2020-10-16 15:38:29 > Request #: 2020-10-11 ℹ 2020-10-16 15:38:30 > Request #: 2020-10-12 ℹ 2020-10-16 15:38:32 > Request #: 2020-10-13 Error in x[[1]] : subscript out of bounds

byBatch executes fine.

MarkEdmondson1234 · 2020-10-16T22:04:32Z

I think only use byBatch for now - the api itself is undergoing changes so I'm waiting for those to settle before publishing any new versions. It could be byDate is not needed anymore.

MarkEdmondson1234 added the bug label Aug 5, 2020

MarkEdmondson1234 mentioned this issue Aug 5, 2020

Batch indexing sometimes misses MarkEdmondson1234/googleAuthR#191

Open

MarkEdmondson1234 mentioned this issue Apr 23, 2021

Batching - Problems with walk_data = "byDate" #67

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error: Error in x[[1]] : subscript out of bounds #62

Error: Error in x[[1]] : subscript out of bounds #62

AdamJayJonca commented Aug 4, 2020

MarkEdmondson1234 commented Aug 4, 2020

AdamJayJonca commented Aug 5, 2020 •

edited

Loading

MarkEdmondson1234 commented Aug 5, 2020 via email

AdamJayJonca commented Aug 5, 2020

MarkEdmondson1234 commented Aug 5, 2020

MarkEdmondson1234 commented Aug 5, 2020

mmueller-cm commented Oct 16, 2020 •

edited

Loading

MarkEdmondson1234 commented Oct 16, 2020

Error: Error in x[[1]] : subscript out of bounds #62

Error: Error in x[[1]] : subscript out of bounds #62

Comments

AdamJayJonca commented Aug 4, 2020

Session Info

MarkEdmondson1234 commented Aug 4, 2020

AdamJayJonca commented Aug 5, 2020 • edited Loading

MarkEdmondson1234 commented Aug 5, 2020 via email

AdamJayJonca commented Aug 5, 2020

MarkEdmondson1234 commented Aug 5, 2020

MarkEdmondson1234 commented Aug 5, 2020

mmueller-cm commented Oct 16, 2020 • edited Loading

MarkEdmondson1234 commented Oct 16, 2020

AdamJayJonca commented Aug 5, 2020 •

edited

Loading

mmueller-cm commented Oct 16, 2020 •

edited

Loading