This project is mirrored from https://github.com/cockroachdb/cockroach. Pull mirroring updated .
  1. 06 May, 2021 18 commits
    • craig[bot]'s avatar
      Merge #62243 #62629 #63841 #64706 #64776 #64777 · c731dc7a
      craig[bot] authored
      62243: forbiddenmethod: add (disabled) lint against  `(*errgroup.Group).Go` and `go` r=knz,erikgrinaker,jordanlewis a=tbg
      
      This helps move the needle on #58164 by introducing linters that
      force both the use of a `ctxgroup` over an `errgroup` and prevent
      direct use of the `go` keyword.
      
      They are disabled by default because we need to fix the issues they
      find first. This can be done (for development) in
      `forbiddenmethod.Analyzers`. They can then also be invoked in a targeted
      fashion:
      
      ```
      go vet -vettool ./bin/roachvet -errgroupgo ./pkg/somewhere
      go vet -vettool ./bin/roachvet -nakedgo ./pkg/somewhere
      ```
      
      Release note: None
      
      62629: docs, server: update auto-generated logging docs r=knz a=taroface
      
      The auto-generated docs at https://github.com/cockroachdb/cockroach/tree/master/docs/generated will be used as includes on the public docs site. This PR makes copyedits and some corrections to the text.
      
      - In the log format docs, I added formatting that I think will add code ticks to field names. In some cases (e.g., ‹...›) I may  not have done this correctly.
      - The structured event description for `node_decommissioning` is not appearing correctly in eventlog.md. The generated sentence for this event does not match the corresponding line in the other event descriptions. I fixed a typo where this was written as "NodeDecommissioned" but I'm not sure if that fixes the issue.
      - In a few cases I pasted lines that were much longer than what was originally in the commented-out blocks. I hope this doesn't break the generation!
      
      Release note: none
      
      63841: pgwire,logpb,eventpb: various structured logging doc updates r=taroface a=knz
      
      First commit from #62629.
      Fixes #63764. 
      Fixes #63762. 
      
      See individual commits for details.
      
      cc @taroface 
      
      64706: pkg/cli: new flag `--log-config-file` to simplify log configuration r=rauchenstein a=knz
      
      Fixes #64349.
      cc @thtruo 
      
      Release note (cli change): The new parameter `--log-config-file`
      simplifies the process of loading the logging configuration from a
      YAML file. Instead of passing the content of the file via
      e.g. `--log=$(cat file.yaml)`, it is now possible to pass the path to
      the file `--log-config-file=file.yaml`.
      
      Note: each occurrence of `--log` and `--log-config-file` on the
      command line overrides the configuration set from previous
      occurrences.
      
      64776: workload: mark tpcc idle-conns flag as runtime-only r=dt a=dt
      
      Closes #64678.
      
      Release note: none.
      
      64777: ccl/backupccl: skip TestBackupWorkerFailure r=stevendanna a=adityamaru
      
      Refs: #64773
      
      Reason: flaky test
      
      Generated by bin/skip-test.
      
      Release justification: non-production code changes
      
      Release note: None
      Co-authored-by: default avatarTobias Grieger <[email protected]>
      Co-authored-by: default avatartaroface <[email protected]>
      Co-authored-by: default avatarRaphael 'kena' Poss <[email protected]>
      Co-authored-by: default avatarDavid Taylor <[email protected]>
      Co-authored-by: default avatarAditya Maru <[email protected]>
      c731dc7a
    • craig[bot]'s avatar
      Merge #64285 · 2b84142d
      craig[bot] authored
      64285: cliccl/debug_backup.go: add `--with-revisions` flag to allow debugging revisions of data r=pbardea a=Elliebababa
      
      This patch adds `--with-revisions` on `debug export` to allow users
      to export revisions of table data. If `--with-revisions` is specified,
      revisions of data are returned to users, with an extra column
      displaying the revision time of that record.
      
      Note: 
      Schema changes make it hard to maintain correct display of columns at different timestamps in one shot,
      So this feature:
      - do not display schema changes across revisions. 
      - display data changes since last schema change only. 
      
      That is, `--with-revisons` only display *part of* revisions history instead of *all* revision history.
      We migitate this by giving message saying `DETECTED SCHEMA CHANGE AT t1, ONLY SHOWING UPDATES IN RANGE [t1,t2]`.
      Then users are able to take exploratory steps to trace back to revision history of data by repeatedly running 
      `cockroach debug backup export <backup_url> --table=<table> --with-revisions --up-to=t1-1`
      
      ----------------------------------------
      
      Example usage:
      ```
      $cockroach debug backup export <backup_url> --table=<table> --with-revisions 
      Data changes happened between t1 and t2 :
      DETECTED SCHEMA CHANGE AT t1, ONLY SHOWING UPDATES IN RANGE [t1,t2]
      1,null,2021-04-22 18:12:47.685284 +0000 UTC
      2,'3rd update',2021-04-22 18:13:41.27284 +0000 UTC
      2,'2nd update',2021-04-22 18:13:34.718665 +0000 UTC
      2,'1st update',2021-04-22 18:13:28.966741 +0000 UTC
      2,null,2021-04-22 18:12:50.053996 +0000 UTC
      ```
      ```
      $cockroach debug backup export <backup_url> --table=<table> --with-revisions --up-to=’2021-04-22 18:13:40’
      DETECTED SCHEMA CHANGE AT t1, ONLY SHOWING UPDATES IN RANGE [t1,2021-04-22 18:13:40]
      1,null,2021-04-22 18:12:47.685284 +0000 UTC
      2,'2nd update',2021-04-22 18:13:34.718665 +0000 UTC
      2,'1st update',2021-04-22 18:13:28.966741 +0000 UTC
      2,null,2021-04-22 18:12:50.053996 +0000 UTC
      ```
      *t1 is the last schema changes before the time specified by --up-to*
      *t2 is the endtime of backup*
      
      --------------------
      
      Release note (cli change): 
      This is an experimenal/beta feature of backup debug tool to
       allow users to export revisions of data from backup. We add 
      `--with-revisions` on `debug export` to allow users to  export 
      revisions of table data. If `--with-revisions` is specified, revisions
      of data are returned to users, with an extra column displaying the 
      revision time of that record.
      Co-authored-by: default avatarelliebababa <[email protected]>
      2b84142d
    • Aditya Maru's avatar
      ccl/backupccl: skip TestBackupWorkerFailure · 7e1744e8
      Aditya Maru authored
      Refs: #64773
      
      Reason: flaky test
      
      Generated by bin/skip-test.
      
      Release justification: non-production code changes
      
      Release note: None
      7e1744e8
    • craig[bot]'s avatar
      Merge #64102 · 42bc2393
      craig[bot] authored
      64102: concurrency: improvements to lock garbage collection r=sumeerbhola a=sumeerbhola
      
      This includes improvements to lock garbage collection, and
      reduces the number of locks by consulting the
      finalizedTxnCache when adding large numbers of discovered
      locks.
      
      Previously, AddDiscoveredLock could result in locks that
      were never garbage collected, since we had a very loose
      mechanism for ensuring liveness of requests. This could
      blow up the lock table size to much higher than the limit.
      Additionally, we have seen (in a roachtest), tryClearLocks
      consuming excessive CPU.
      
      This makes the following improvements to garbage collection:
      - At most one lock per Request, added by AddDiscoveredLock,
        is transiently not removable. Even that lock becomes
        removable once the request does another ScanAndEnqueue
        or Dequeues. This should allow most locks to be garbage
        collected.
      - We no longer clear all locks when the max size is exceeded,
        and instead go down to max locks/2. This preserves some
        fidelity wrt ordering, while avoiding the cost of frequent
        enforcement.
      - Enforcement is throttled such that it happens in increments
        of 5% of the max count, so that if we end up in a situation
        where most locks are not removable, we reduce the amount of
        cpu spent in unsuccessful enforcement.
      
      To prevent huge numbers of discoverd locks from already
      finalized transactions from being added to the lock table,
      the finalizedTxnCache is consulted above a certain threshold
      of discovered lock count. When below that threshold, we
      continue to prefer to add the lock and consult the
      finalizedTxnCache in the following call to ScanAndEnqueue,
      since that provides some coordination that de-duplicates
      intent resolution.
      
      Fixes #62470
      
      Release note (performance improvement): Peak memory usage in
      the lock table is significantly reduced. Runaway CPU usage due
      to wasted quadratic time complexity in clearing unclearable
      locks is addressed.
      Co-authored-by: default avatarsumeerbhola <[email protected]>
      42bc2393
    • David Taylor's avatar
      workload: mark tpcc idle-conns flag as runtime-only · d1a3a7c0
      David Taylor authored
      Closes #64678.
      
      Release note: none.
      d1a3a7c0
    • Raphael 'kena' Poss's avatar
      docs: mention the full table scan setting for SQL_PERF events · 04cd49bd
      Raphael 'kena' Poss authored
      The `SlowQuery` event only gets logged for full table scans
      if a certain cluster setting is set. This commit explains this.
      
      Release note: None
      04cd49bd
    • Raphael 'kena' Poss's avatar
      logpb: don't mention 'pgwire events' under SQL_EXEC. Mention panics. · a3d00243
      Raphael 'kena' Poss authored
      The SQL_EXEC channel contains statement execution events (structured)
      and a copy of Go panics encountered during SQL execution (unstructured).
      
      Release note: None
      a3d00243
    • Raphael 'kena' Poss's avatar
      pgwire: log HBA parsing errors to the OPS channel · f5160ddb
      Raphael 'kena' Poss authored
      Release note (security update): Syntax errors in the HBA configuration
      in cluster setting `server.host_based_authentication.configuration`
      are now logged on the OPS channel.
      f5160ddb
    • taroface's avatar
      docs, server: update auto-generated logging docs · 1f74b87b
      taroface authored
      Release note: None
      Co-authored-by: default avatarRaphael 'kena' Poss <[email protected]>
      1f74b87b
    • craig[bot]'s avatar
      Merge #64737 · 1cde28a5
      craig[bot] authored
      64737: storage/cloud: remove gcs key cluster setting r=dt a=dt
      
      This was deprecated in 21.1 because it does make sense in multi-user/multi-tenant
      clusters. This change removes it.
      
      This also changes the behavior of the AUTH option in google cloud storage URIs:
      Previously AUTH=specified was required to use passed credentials, while
      no value for AUTH or the value 'default' would attempt to use the value saved
      in the cluster setting cloudstorage.gs.default.key. AUTH=default would fail if
      this value was not set, while an empty value of AUTH and an empty setting would
      fall back to AUTH=implicit. After this change, AUTH=implicit is the only configuration
      in which implicit (node's role / machine account / env) will be used, and
      any other value for AUTH, including unset or specified, will use credentials
      from the CREDENTIALS param, or return an error if it is not set.
      
      Release note (backward-incompatible change): The deprecated setting cloudstorage.gs.default.key has been removed, and the behavior of the 'AUTH' param in Google Cloud Storage BACKUP and IMPORT URIs has been changed, with the default behavior now being that of 'AUTH=specified' which uses the credentials passed in the CREDENTIALS parameter, and the previous default behavior of using the node's implicit access (via its machine account or role) now requires explicitly passing 'AUTH=implicit'.
      Co-authored-by: default avatarDavid Taylor <[email protected]>
      1cde28a5
    • Tobias Grieger's avatar
      forbiddenmethod: add (disabled) lint against `(*errgroup.Group).Go` and `go` · 6d69977c
      Tobias Grieger authored
      This helps move the needle on #58164 by introducing linters that
      force both the use of a `ctxgroup` over an `errgroup` and prevent
      direct use of the `go` keyword.
      
      They are disabled by default because we need to fix the issues they
      find first. This can be done (for development) in
      `forbiddenmethod.Analyzers`. They can then also be invoked in a targeted
      fashion:
      
      ```
      go vet -vettool ./bin/roachvet -errgroupgo ./pkg/somewhere
      go vet -vettool ./bin/roachvet -nakedgo ./pkg/somewhere
      ```
      
      Release note: None
      6d69977c
    • craig[bot]'s avatar
      Merge #63383 · 72a144d8
      craig[bot] authored
      63383: backupccl: add timeout to ExportRequest sent by a backup r=pbardea a=adityamaru
      
      Previously, there was no timeout around the ExportRequest that
      was sent as part of a backup. If the request returned certain
      kinds of unambiguous errors, dist sender would keep retrying the request
      thereby causing the backup to hang.
      
      This change adds an upper bound on the time it should take the sender
      to receive a response, thereby erroring out instead of mysteriously
      hanging. The upper bound is configurable via a cluster settings and is
      set to 5 mins by default, which is very conservative as these requests
      should be very fast under normal circumstances.
      
      Release note: None
      Co-authored-by: default avatarAditya Maru <[email protected]>
      72a144d8
    • craig[bot]'s avatar
      Merge #63736 · d21ad0e3
      craig[bot] authored
      63736: kvserver: add a test to verify lease preferences during an outage r=lunevalex a=lunevalex
      
      In #62485 we observed that a datacenter that does not have the lease
      preference can acquire a lease to a range when the leaseholder dies.
      This happens because we prioritize availibility over lease preferences,
      so any node with a replica that detects that a range does not have a valid
      lease will try to steal it. This test verifies that this is possible and
      that upon lease acquisition the new lease holder will upreplicate the range
      amd immediatly give up the lease to the preffered lease holder if possible.
      
      Release note: None
      Co-authored-by: default avatarAlex Lunev <[email protected]>
      d21ad0e3
    • craig[bot]'s avatar
      Merge #64696 · 97e3a2ef
      craig[bot] authored
      64696: roachtest: add ternary logic partitioning (TLP) test r=mgartner a=mgartner
      
      #### roachtest: clarify SQLSmith comment
      
      Release note: None
      
      #### roachtest: add ternary logic partitioning (TLP) test
      
      This commit adds a roachtest that performs ternary logic partitioning
      (TLP) testing. TLP is a method for logically testing a database which is
      based on the logical guarantee that for a given predicate `p`, all rows
      must satisfy exactly one of the following three predicates: `p`, `NOT
      p`, `p IS NULL`. Unioning the results of all three "partitions" should
      yield the same result as an "unpartitioned" query with a `true`
      predicate.
      
      TLP is implemented in [SQLancer](https://github.com/sqlancer/sqlancer)
      and more information can be found at
      https://www.manuelrigger.at/preprints/TLP.pdf.
      
      We currently implement a limited form of TLP that only runs queries of
      the form `SELECT * FROM table WHERE <predicate>` where `<predicate>` is
      randomly generated. We also only verify that the number of rows returned
      by the unpartitioned and partitioned queries are equal, not that the
      values of the rows are equal. See the documentation for
      `Smither.GenerateTLP` for more details.
      
      Fixes #62900
      
      Release note: None
      Co-authored-by: default avatarMarcus Gartner <[email protected]>
      97e3a2ef
    • Marcus Gartner's avatar
      roachtest: add ternary logic partitioning (TLP) test · 777382e6
      Marcus Gartner authored
      This commit adds a roachtest that performs ternary logic partitioning
      (TLP) testing. TLP is a method for logically testing a database which is
      based on the logical guarantee that for a given predicate `p`, all rows
      must satisfy exactly one of the following three predicates: `p`, `NOT
      p`, `p IS NULL`. Unioning the results of all three "partitions" should
      yield the same result as an "unpartitioned" query with a `true`
      predicate.
      
      TLP is implemented in [SQLancer](https://github.com/sqlancer/sqlancer)
      and more information can be found at
      https://www.manuelrigger.at/preprints/TLP.pdf.
      
      We currently implement a limited form of TLP that only runs queries of
      the form `SELECT * FROM table WHERE <predicate>` where `<predicate>` is
      randomly generated. We also only verify that the number of rows returned
      by the unpartitioned and partitioned queries are equal, not that the
      values of the rows are equal. See the documentation for
      `Smither.GenerateTLP` for more details.
      
      Release note: None
      777382e6
    • Marcus Gartner's avatar
      roachtest: clarify SQLSmith comment · fd25b690
      Marcus Gartner authored
      Release note: None
      fd25b690
    • craig[bot]'s avatar
      Merge #64746 · 6ca29ddc
      craig[bot] authored
      64746: colfetcher: fix recent bug with cfetcher r=yuzefovich a=yuzefovich
      
      2223a601 intended to set all not needed
      vectors to all null values in the cFetcher, but one case was overlooked.
      This commit fixes that problem.
      
      Fixes: #64676.
      
      Release note: None (no release with this bug)
      Co-authored-by: default avatarYahor Yuzefovich <[email protected]>
      6ca29ddc
    • sumeerbhola's avatar
      concurrency: improvements to lock table memory management · 892a4a32
      sumeerbhola authored
      This includes improvements to lock garbage collection, and
      reduces the number of locks by consulting the
      finalizedTxnCache when adding large numbers of discovered
      locks.
      
      Previously, AddDiscoveredLock could result in locks that
      were never garbage collected, since we had a very loose
      mechanism for ensuring liveness of requests. This could
      blow up the lock table size to much higher than the limit.
      Additionally, we have seen (in a roachtest), tryClearLocks
      consuming excessive CPU.
      
      This makes the following improvements to garbage collection:
      - At most one lock per Request, added by AddDiscoveredLock,
        is transiently not removable. Even that lock becomes
        removable once the request does another ScanAndEnqueue
        or Dequeues. This should allow most locks to be garbage
        collected.
      - We no longer clear all locks when the max size is exceeded,
        and instead go down to max locks/2. This preserves some
        fidelity wrt ordering, while avoiding the cost of frequent
        enforcement.
      - Enforcement is throttled such that it happens in increments
        of 5% of the max count, so that if we end up in a situation
        where most locks are not removable, we reduce the amount of
        cpu spent in unsuccessful enforcement.
      
      To prevent huge numbers of discoverd locks from already
      finalized transactions from being added to the lock table,
      the finalizedTxnCache is consulted above a certain threshold
      of discovered lock count. When below that threshold, we
      continue to prefer to add the lock and consult the
      finalizedTxnCache in the following call to ScanAndEnqueue,
      since that provides some coordination that de-duplicates
      intent resolution.
      
      Fixes #62470
      
      Release note (performance improvement): Peak memory usage in
      the lock table is significantly reduced. Runaway CPU usage due
      to wasted quadratic time complexity in clearing unclearable
      locks is addressed.
      892a4a32
  2. 05 May, 2021 22 commits
    • craig[bot]'s avatar
      Merge #64731 · 5c938015
      craig[bot] authored
      64731: add missing Source button syntax r=taroface a=taroface
      
      The Source button syntax went missing at some point, causing the button icon to disappear. This PR adds the missing syntax.
      Co-authored-by: default avatarRyan Kuo <[email protected]>
      5c938015
    • craig[bot]'s avatar
      Merge #64035 · 124ccc9e
      craig[bot] authored
      64035: sql: adding new tables at pg_catalog r=rafiss a=mnovelodou
      
      Previously, we were unable to add this tables because its
      types were unimplemented
      This was inadequate because some these unimplemented types
      could be mapped to other implemented types
      To address this, this patch adds these tables by mapping
      unimplemented types.
      
      Release note (sql): Added new tables at pg_catalog:
      - pg_partitioned_table
      - pg_replication_origin_status
      - pg_init_privs
      - pg_replication_slots
      - pg_policy
      - pg_sequences
      - pg_subscription_rel
      - pg_largeobject_metadata
      Co-authored-by: default avatarMiguelNovelo <[email protected]>
      124ccc9e
    • Yahor Yuzefovich's avatar
      colfetcher: fix recent bug with cfetcher · ef046410
      Yahor Yuzefovich authored
      2223a601 intended to set all not needed
      vectors to all null values in the cFetcher, but one case was overlooked.
      This commit fixes that problem.
      
      Release note: None (no release with this bug)
      ef046410
    • David Taylor's avatar
      storage/cloud: remove gcs key cluster setting · 3912f2e5
      David Taylor authored
      This was deprecated in 21.1 because it does make sense in multi-user/multi-tenant
      clusters. This change removes it.
      
      This also changes the behavior of the AUTH option in google cloud storage URIs:
      Previously AUTH=specified was required to use passed credentials, while
      no value for AUTH or the value 'default' would attempt to use the value saved
      in the cluster setting cloudstorage.gs.default.key. AUTH=default would fail if
      this value was not set, while an empty value of AUTH and an empty setting would
      fall back to AUTH=implicit. After this change, AUTH=implicit is the only configuration
      in which implicit (node's role / machine account / env) will be used, and
      any other value for AUTH, including unset or specified, will use credentials
      from the CREDENTIALS param, or return an error if it is not set.
      
      Release note (backward-incompatible change): The deprecated setting cloudstorage.gs.default.key has been removed, and the behavior of the 'AUTH' param in Google Cloud Storage BACKUP and IMPORT URIs has been changed, with the default behavior now being that of 'AUTH=specified' which uses the credentials passed in the CREDENTIALS parameter, and the previous default behavior of using the node's implicit access (via its machine account or role) now requires explicitly passing 'AUTH=implicit'.
      3912f2e5
    • craig[bot]'s avatar
      Merge #64563 · 6fb13b63
      craig[bot] authored
      64563: kvserver: clean up miscellaneous comments over `replicaRankings` r=aayushshah15 a=aayushshah15
      
      Release note: None
      Co-authored-by: default avatarAayush Shah <[email protected]>
      6fb13b63
    • craig[bot]'s avatar
      Merge #61401 #64708 · 87d7c4e6
      craig[bot] authored
      61401: storage: coordinated iteration with limits in intentInterleavin… r=sumeerbhola a=sumeerbhola
      
      …gIter
      
      intentInterleavingIter uses the *WithLimit() functions exposed
      by pebble.Iterator.
      
      This is motivated by the O(N^2) complexity we observe when doing
      coordinated iteration over the lock table and MVCC keys where
      all the intents in the lock table are deleted but have not yet
      been compacted away. The O(N^2) complexity arises in two cases:
      - Limited scans: the extreme is a limited scan with a
        limit of 1. Each successive scan traverses N, N-1, ...
        deleted locks.
      - Mix of forward/reverse iteration in a single scan: A Next/SeekGE
        will incur a cost of N and the Prev/SeekLT another cost of N.
        This situation could be potentially fixed with a narrower
        solution, given the constrained way that the combination of
        forward and backward iteration is used in pebbleMVCCScanner
        (forward iteration using Next and SeekGE when doing a reverse
        scan), however the solution is likely to be fragile since it
        would require the code in intentInterleavingIter to know how
        it is used in pebbleMVCCScanner. The limited iteration mode
        solves it in a more general way.
      
      By placing a limit when doing such coordinated iteration, we
      restore iteration to O(N) for N keys.
      
      Some preliminary benchmark numbers are included -- old is master.
      Note the speedup in the `ScanOneAllIntentsResolved/separated=true`
      cases which were taking many ms. The remaining are effectively
      unchanged (these are from single runs, so some of the deltas
      there may be noise).
      
      Note that there is a shift
      in where comparisons get performed in some of these benchmarks:
      the comparison i.intentKey.Compare(i.iterKey.Key) in
      intentInterleavingIter is replaced (when there is a live intent
      for a different key) with a corresponding comparison in
      pebble.Iterator's find{Next,Prev}Entry comparing with the limit
      parameter. However we are still paying the cost for extra
      comparisons when there are only dead intents within the iterator
      bounds: say there are N keys each with M versions, so N\*M dead
      intents each with a SINGLEDEL,SET pair (so 2\*N\*M keys in Pebble,
      of which N\*M are considered distinct UserKeys). There will
      eventually be N calls to pebble.Iterator.findNextEntry (when
      iterating forward), each of which will call Iterator.nextUserKey
      M times, after each of which it will do a comparison with the
      limit key. So a total of N\*M extra comparisons to iterate over
      these 2\*N\*M keys in Pebble. The cost of these seems negligible
      based on the performance numbers below, which are not with
      a deep mergingIter stack in Pebble -- a more realistic setting
      with a deep mergingIter stack will have more comparisons for
      merging across iterators, which should further decrease the
      relative performance effect of the N\*M comparisons we are
      paying in the top-level pebble.Iterator.
      
      ```
      name                                                                           old time/op    new time/op    delta
      ScanOneAllIntentsResolved/separated=false/versions=200/percent-flushed=0-16      51.1µs ± 0%    49.9µs ± 0%    -2.26%
      ScanOneAllIntentsResolved/separated=false/versions=200/percent-flushed=50-16     21.9µs ± 0%    24.2µs ± 0%   +10.52%
      ScanOneAllIntentsResolved/separated=false/versions=200/percent-flushed=90-16     5.52µs ± 0%    5.50µs ± 0%    -0.36%
      ScanOneAllIntentsResolved/separated=false/versions=200/percent-flushed=100-16    2.37µs ± 0%    2.15µs ± 0%    -9.04%
      ScanOneAllIntentsResolved/separated=true/versions=200/percent-flushed=0-16       47.4ms ± 0%     0.1ms ± 0%   -99.88%
      ScanOneAllIntentsResolved/separated=true/versions=200/percent-flushed=50-16      11.1ms ± 0%     0.0ms ± 0%   -99.81%
      ScanOneAllIntentsResolved/separated=true/versions=200/percent-flushed=90-16      1.07ms ± 0%    0.01ms ± 0%   -99.48%
      ScanOneAllIntentsResolved/separated=true/versions=200/percent-flushed=100-16     2.08µs ± 0%    2.04µs ± 0%    -1.88%
      
      IntentScan/separated=false/versions=10/percent-flushed=0-16                      2.96µs ± 0%    2.80µs ± 0%    -5.67%
      IntentScan/separated=false/versions=10/percent-flushed=50-16                     1.93µs ± 0%    1.80µs ± 0%    -6.78%
      IntentScan/separated=false/versions=10/percent-flushed=80-16                     1.55µs ± 0%    1.46µs ± 0%    -5.73%
      IntentScan/separated=false/versions=10/percent-flushed=90-16                     1.22µs ± 0%    1.20µs ± 0%    -2.05%
      IntentScan/separated=false/versions=10/percent-flushed=100-16                     937ns ± 0%     864ns ± 0%    -7.79%
      IntentScan/separated=false/versions=100/percent-flushed=0-16                     27.0µs ± 0%    19.8µs ± 0%   -26.65%
      IntentScan/separated=false/versions=100/percent-flushed=50-16                    13.3µs ± 0%     9.2µs ± 0%   -31.13%
      IntentScan/separated=false/versions=100/percent-flushed=80-16                    4.06µs ± 0%    3.71µs ± 0%    -8.72%
      IntentScan/separated=false/versions=100/percent-flushed=90-16                    2.69µs ± 0%    2.59µs ± 0%    -3.93%
      IntentScan/separated=false/versions=100/percent-flushed=100-16                    985ns ± 0%     931ns ± 0%    -5.48%
      IntentScan/separated=false/versions=200/percent-flushed=0-16                     50.1µs ± 0%    50.9µs ± 0%    +1.76%
      IntentScan/separated=false/versions=200/percent-flushed=50-16                    26.7µs ± 0%    19.7µs ± 0%   -26.19%
      IntentScan/separated=false/versions=200/percent-flushed=80-16                    8.76µs ± 0%    6.30µs ± 0%   -28.06%
      IntentScan/separated=false/versions=200/percent-flushed=90-16                    4.18µs ± 0%    3.83µs ± 0%    -8.33%
      IntentScan/separated=false/versions=200/percent-flushed=100-16                   1.04µs ± 0%    0.95µs ± 0%    -8.47%
      IntentScan/separated=false/versions=400/percent-flushed=0-16                     22.2µs ± 0%    15.5µs ± 0%   -30.26%
      IntentScan/separated=false/versions=400/percent-flushed=50-16                    9.76µs ± 0%    6.91µs ± 0%   -29.22%
      IntentScan/separated=false/versions=400/percent-flushed=80-16                    20.4µs ± 0%    12.3µs ± 0%   -39.94%
      IntentScan/separated=false/versions=400/percent-flushed=90-16                    9.09µs ± 0%    6.29µs ± 0%   -30.87%
      IntentScan/separated=false/versions=400/percent-flushed=100-16                   1.79µs ± 0%    1.68µs ± 0%    -6.35%
      IntentScan/separated=true/versions=10/percent-flushed=0-16                       3.47µs ± 0%    3.33µs ± 0%    -4.21%
      IntentScan/separated=true/versions=10/percent-flushed=50-16                      1.93µs ± 0%    1.83µs ± 0%    -5.17%
      IntentScan/separated=true/versions=10/percent-flushed=80-16                      1.38µs ± 0%    1.49µs ± 0%    +8.15%
      IntentScan/separated=true/versions=10/percent-flushed=90-16                      1.20µs ± 0%    1.36µs ± 0%   +13.36%
      IntentScan/separated=true/versions=10/percent-flushed=100-16                      923ns ± 0%    1002ns ± 0%    +8.56%
      IntentScan/separated=true/versions=100/percent-flushed=0-16                      25.3µs ± 0%    28.9µs ± 0%   +14.20%
      IntentScan/separated=true/versions=100/percent-flushed=50-16                     6.74µs ± 0%    7.27µs ± 0%    +7.88%
      IntentScan/separated=true/versions=100/percent-flushed=80-16                     3.43µs ± 0%    3.69µs ± 0%    +7.43%
      IntentScan/separated=true/versions=100/percent-flushed=90-16                     2.31µs ± 0%    2.60µs ± 0%   +12.38%
      IntentScan/separated=true/versions=100/percent-flushed=100-16                     973ns ± 0%    1056ns ± 0%    +8.53%
      IntentScan/separated=true/versions=200/percent-flushed=0-16                      46.9µs ± 0%    51.8µs ± 0%   +10.41%
      IntentScan/separated=true/versions=200/percent-flushed=50-16                     14.7µs ± 0%    13.8µs ± 0%    -6.30%
      IntentScan/separated=true/versions=200/percent-flushed=80-16                     5.70µs ± 0%    6.04µs ± 0%    +6.02%
      IntentScan/separated=true/versions=200/percent-flushed=90-16                     3.47µs ± 0%    3.69µs ± 0%    +6.52%
      IntentScan/separated=true/versions=200/percent-flushed=100-16                    0.99µs ± 0%    1.06µs ± 0%    +6.83%
      IntentScan/separated=true/versions=400/percent-flushed=0-16                      32.4µs ± 0%    26.3µs ± 0%   -18.79%
      IntentScan/separated=true/versions=400/percent-flushed=50-16                     37.4µs ± 0%    38.9µs ± 0%    +4.03%
      IntentScan/separated=true/versions=400/percent-flushed=80-16                     12.7µs ± 0%    11.8µs ± 0%    -6.98%
      IntentScan/separated=true/versions=400/percent-flushed=90-16                     5.56µs ± 0%    6.19µs ± 0%   +11.30%
      IntentScan/separated=true/versions=400/percent-flushed=100-16                    1.73µs ± 0%    1.87µs ± 0%    +8.23%
      ScanAllIntentsResolved/separated=false/versions=200/percent-flushed=0-16         46.7µs ± 0%    47.9µs ± 0%    +2.62%
      ScanAllIntentsResolved/separated=false/versions=200/percent-flushed=50-16        19.8µs ± 0%    19.2µs ± 0%    -3.20%
      ScanAllIntentsResolved/separated=false/versions=200/percent-flushed=90-16        3.83µs ± 0%    3.89µs ± 0%    +1.62%
      ScanAllIntentsResolved/separated=false/versions=200/percent-flushed=100-16        972ns ± 0%     993ns ± 0%    +2.16%
      ScanAllIntentsResolved/separated=true/versions=200/percent-flushed=0-16          63.1µs ± 0%    59.6µs ± 0%    -5.55%
      ScanAllIntentsResolved/separated=true/versions=200/percent-flushed=50-16         17.0µs ± 0%    15.1µs ± 0%   -10.87%
      ScanAllIntentsResolved/separated=true/versions=200/percent-flushed=90-16         3.26µs ± 0%    3.61µs ± 0%   +10.73%
      ScanAllIntentsResolved/separated=true/versions=200/percent-flushed=100-16         918ns ± 0%     945ns ± 0%    +2.94%
      IntentResolution/separated=false/versions=10/percent-flushed=0-16                2.25µs ± 0%    2.23µs ± 0%    -1.15%
      IntentResolution/separated=false/versions=10/percent-flushed=50-16               3.09µs ± 0%    2.89µs ± 0%    -6.53%
      IntentResolution/separated=false/versions=10/percent-flushed=80-16               2.73µs ± 0%    2.44µs ± 0%   -10.64%
      IntentResolution/separated=false/versions=10/percent-flushed=90-16               1.90µs ± 0%    1.88µs ± 0%    -0.95%
      IntentResolution/separated=false/versions=10/percent-flushed=100-16              2.45µs ± 0%    2.41µs ± 0%    -1.59%
      IntentResolution/separated=false/versions=100/percent-flushed=0-16               4.59µs ± 0%    4.70µs ± 0%    +2.42%
      IntentResolution/separated=false/versions=100/percent-flushed=50-16              3.83µs ± 0%    3.91µs ± 0%    +2.01%
      IntentResolution/separated=false/versions=100/percent-flushed=80-16              3.18µs ± 0%    3.14µs ± 0%    -1.23%
      IntentResolution/separated=false/versions=100/percent-flushed=90-16              3.19µs ± 0%    3.06µs ± 0%    -4.20%
      IntentResolution/separated=false/versions=100/percent-flushed=100-16             2.65µs ± 0%    2.34µs ± 0%   -11.71%
      IntentResolution/separated=false/versions=200/percent-flushed=0-16               5.76µs ± 0%    5.70µs ± 0%    -1.04%
      IntentResolution/separated=false/versions=200/percent-flushed=50-16              4.09µs ± 0%    4.01µs ± 0%    -2.05%
      IntentResolution/separated=false/versions=200/percent-flushed=80-16              3.22µs ± 0%    3.15µs ± 0%    -2.05%
      IntentResolution/separated=false/versions=200/percent-flushed=90-16              3.14µs ± 0%    3.17µs ± 0%    +0.67%
      IntentResolution/separated=false/versions=200/percent-flushed=100-16             2.54µs ± 0%    2.50µs ± 0%    -1.81%
      IntentResolution/separated=false/versions=400/percent-flushed=0-16               2.83µs ± 0%    2.94µs ± 0%    +3.74%
      IntentResolution/separated=false/versions=400/percent-flushed=50-16              2.75µs ± 0%    2.71µs ± 0%    -1.74%
      IntentResolution/separated=false/versions=400/percent-flushed=80-16              2.84µs ± 0%    2.83µs ± 0%    -0.14%
      IntentResolution/separated=false/versions=400/percent-flushed=90-16              2.78µs ± 0%    2.72µs ± 0%    -2.09%
      IntentResolution/separated=false/versions=400/percent-flushed=100-16             2.04µs ± 0%    2.03µs ± 0%    -0.39%
      IntentResolution/separated=true/versions=10/percent-flushed=0-16                 3.03µs ± 0%    3.07µs ± 0%    +1.15%
      IntentResolution/separated=true/versions=10/percent-flushed=50-16                3.21µs ± 0%    3.24µs ± 0%    +0.84%
      IntentResolution/separated=true/versions=10/percent-flushed=80-16                2.88µs ± 0%    2.77µs ± 0%    -3.81%
      IntentResolution/separated=true/versions=10/percent-flushed=90-16                2.67µs ± 0%    2.67µs ± 0%    -0.04%
      IntentResolution/separated=true/versions=10/percent-flushed=100-16               2.21µs ± 0%    2.22µs ± 0%    +0.32%
      IntentResolution/separated=true/versions=100/percent-flushed=0-16                8.16µs ± 0%    8.36µs ± 0%    +2.44%
      IntentResolution/separated=true/versions=100/percent-flushed=50-16               4.18µs ± 0%    3.98µs ± 0%    -4.69%
      IntentResolution/separated=true/versions=100/percent-flushed=80-16               4.03µs ± 0%    4.10µs ± 0%    +1.76%
      IntentResolution/separated=true/versions=100/percent-flushed=90-16               3.74µs ± 0%    3.78µs ± 0%    +1.23%
      IntentResolution/separated=true/versions=100/percent-flushed=100-16              2.20µs ± 0%    2.26µs ± 0%    +2.77%
      IntentResolution/separated=true/versions=200/percent-flushed=0-16                13.2µs ± 0%    12.8µs ± 0%    -3.25%
      IntentResolution/separated=true/versions=200/percent-flushed=50-16               4.37µs ± 0%    4.28µs ± 0%    -1.90%
      IntentResolution/separated=true/versions=200/percent-flushed=80-16               4.23µs ± 0%    4.38µs ± 0%    +3.62%
      IntentResolution/separated=true/versions=200/percent-flushed=90-16               3.98µs ± 0%    4.08µs ± 0%    +2.44%
      IntentResolution/separated=true/versions=200/percent-flushed=100-16              2.35µs ± 0%    2.28µs ± 0%    -3.15%
      IntentResolution/separated=true/versions=400/percent-flushed=0-16                4.64µs ± 0%    4.59µs ± 0%    -0.93%
      IntentResolution/separated=true/versions=400/percent-flushed=50-16               5.06µs ± 0%    4.77µs ± 0%    -5.74%
      IntentResolution/separated=true/versions=400/percent-flushed=80-16               3.91µs ± 0%    3.78µs ± 0%    -3.32%
      IntentResolution/separated=true/versions=400/percent-flushed=90-16               3.54µs ± 0%    3.50µs ± 0%    -1.05%
      IntentResolution/separated=true/versions=400/percent-flushed=100-16              2.24µs ± 0%    2.28µs ± 0%    +2.01%
      IntentRangeResolution/separated=false/versions=10/percent-flushed=0-16            265µs ± 0%     277µs ± 0%    +4.64%
      IntentRangeResolution/separated=false/versions=10/percent-flushed=50-16           211µs ± 0%     217µs ± 0%    +3.19%
      IntentRangeResolution/separated=false/versions=10/percent-flushed=80-16           202µs ± 0%     209µs ± 0%    +3.61%
      IntentRangeResolution/separated=false/versions=10/percent-flushed=90-16           197µs ± 0%     201µs ± 0%    +1.87%
      IntentRangeResolution/separated=false/versions=10/percent-flushed=100-16          150µs ± 0%     154µs ± 0%    +3.07%
      IntentRangeResolution/separated=false/versions=100/percent-flushed=0-16           528µs ± 0%     531µs ± 0%    +0.52%
      IntentRangeResolution/separated=false/versions=100/percent-flushed=50-16          311µs ± 0%     302µs ± 0%    -2.88%
      IntentRangeResolution/separated=false/versions=100/percent-flushed=80-16          232µs ± 0%     230µs ± 0%    -0.94%
      IntentRangeResolution/separated=false/versions=100/percent-flushed=90-16          227µs ± 0%     226µs ± 0%    -0.35%
      IntentRangeResolution/separated=false/versions=100/percent-flushed=100-16         164µs ± 0%     161µs ± 0%    -2.31%
      IntentRangeResolution/separated=false/versions=200/percent-flushed=0-16           980µs ± 0%     616µs ± 0%   -37.09%
      IntentRangeResolution/separated=false/versions=200/percent-flushed=50-16          396µs ± 0%     323µs ± 0%   -18.36%
      IntentRangeResolution/separated=false/versions=200/percent-flushed=80-16          263µs ± 0%     249µs ± 0%    -5.28%
      IntentRangeResolution/separated=false/versions=200/percent-flushed=90-16          287µs ± 0%     236µs ± 0%   -17.86%
      IntentRangeResolution/separated=false/versions=200/percent-flushed=100-16         161µs ± 0%     164µs ± 0%    +2.08%
      IntentRangeResolution/separated=false/versions=400/percent-flushed=0-16           263µs ± 0%     267µs ± 0%    +1.67%
      IntentRangeResolution/separated=false/versions=400/percent-flushed=50-16          254µs ± 0%     257µs ± 0%    +1.37%
      IntentRangeResolution/separated=false/versions=400/percent-flushed=80-16          257µs ± 0%     259µs ± 0%    +0.94%
      IntentRangeResolution/separated=false/versions=400/percent-flushed=90-16          246µs ± 0%     256µs ± 0%    +4.22%
      IntentRangeResolution/separated=false/versions=400/percent-flushed=100-16         172µs ± 0%     175µs ± 0%    +1.46%
      IntentRangeResolution/separated=true/versions=10/percent-flushed=0-16             443µs ± 0%     447µs ± 0%    +0.77%
      IntentRangeResolution/separated=true/versions=10/percent-flushed=50-16            295µs ± 0%     287µs ± 0%    -2.64%
      IntentRangeResolution/separated=true/versions=10/percent-flushed=80-16            241µs ± 0%     254µs ± 0%    +5.27%
      IntentRangeResolution/separated=true/versions=10/percent-flushed=90-16            238µs ± 0%     240µs ± 0%    +0.82%
      IntentRangeResolution/separated=true/versions=10/percent-flushed=100-16           220µs ± 0%     225µs ± 0%    +2.17%
      IntentRangeResolution/separated=true/versions=100/percent-flushed=0-16           1.84ms ± 0%    2.04ms ± 0%   +10.39%
      IntentRangeResolution/separated=true/versions=100/percent-flushed=50-16           554µs ± 0%     583µs ± 0%    +5.11%
      IntentRangeResolution/separated=true/versions=100/percent-flushed=80-16           377µs ± 0%     389µs ± 0%    +3.31%
      IntentRangeResolution/separated=true/versions=100/percent-flushed=90-16           309µs ± 0%     316µs ± 0%    +2.30%
      IntentRangeResolution/separated=true/versions=100/percent-flushed=100-16          225µs ± 0%     235µs ± 0%    +4.27%
      IntentRangeResolution/separated=true/versions=200/percent-flushed=0-16           3.59ms ± 0%    3.84ms ± 0%    +7.21%
      IntentRangeResolution/separated=true/versions=200/percent-flushed=50-16           875µs ± 0%     998µs ± 0%   +14.00%
      IntentRangeResolution/separated=true/versions=200/percent-flushed=80-16           508µs ± 0%     525µs ± 0%    +3.33%
      IntentRangeResolution/separated=true/versions=200/percent-flushed=90-16           382µs ± 0%     391µs ± 0%    +2.22%
      IntentRangeResolution/separated=true/versions=200/percent-flushed=100-16          266µs ± 0%     240µs ± 0%    -9.75%
      IntentRangeResolution/separated=true/versions=400/percent-flushed=0-16           1.54ms ± 0%    1.69ms ± 0%    +9.22%
      IntentRangeResolution/separated=true/versions=400/percent-flushed=50-16          1.93ms ± 0%    2.13ms ± 0%   +10.45%
      IntentRangeResolution/separated=true/versions=400/percent-flushed=80-16           769µs ± 0%     829µs ± 0%    +7.84%
      IntentRangeResolution/separated=true/versions=400/percent-flushed=90-16           506µs ± 0%     535µs ± 0%    +5.61%
      IntentRangeResolution/separated=true/versions=400/percent-flushed=100-16          248µs ± 0%     252µs ± 0%    +1.52%
      ```
      
      Release justification: Change is localized to intentInterleavingIter
      and pebble.Iterator. It may be risky given it is involved in all reads
      of MVCC data. One possibility would be to adopt the change since it
      only really affects separated intents (the intentIter is the only one
      using the limited iteration), which would unblock turning on separated
      intents in a randomized manner for SQL tests and roachtests, while leaving
      it off by default for 21.1. This would reduce the possibility of code
      rot for separated intents.
      
      Release note: None
      
      64708: roachtest: own `quit-all-nodes` to Server r=knz a=tbg
      
      Release note: None
      Co-authored-by: default avatarsumeerbhola <[email protected]>
      Co-authored-by: default avatarTobias Grieger <[email protected]>
      87d7c4e6
    • Ryan Kuo's avatar
      add missing Source button syntax · b99af4d3
      Ryan Kuo authored
      b99af4d3
    • elliebababa's avatar
      cliccl/debug_backup.go: add `--with-revisions` flag to allow exporting revisions of data · 75f22bcf
      elliebababa authored
      This patch adds `--with-revisions` on `debug export` to allow users
      to export revisions of table data. If `--with-revisions` is specified,
      revisions of data are returned to users, with an extra column
      displaying the revision time of that record.
      
      Release note (cli change): This is an experimenal/beta feature of backup debug
      tool to allow users to export revisions of data from backup. We add `--with-revisions`
      on `debug export` to allow users to export revisions of table data.
      If `--with-revisions` is specified, revisions of data are returned to users,
      with an extra column displaying the revision time of that record.
      75f22bcf
    • craig[bot]'s avatar
      Merge #64662 #64695 #64698 #64718 · 8aabb6db
      craig[bot] authored
      64662: storage/cloud: delegate redaction and implicit auth checks r=dt a=dt
      
      storage/cloud: make implicit auth check funciton of conf
      
      Previously we had two methods that dug through ExternalStorage URI params:
      One, which extracted the params to create a structured config that would
      actually be used to open and interact with that storage, and a second that
      looked for authorization-releated params to decide if access was explicitly
      or implicitly authorized (i.e. had credentials specified, or made use of the
      access the node performing the action had).
      
      This changes the latter to instead be a function of the former, so there is
      only one method that extracts parameters from string URIs to produce a structured
      configuration object, and then the determination of if a given configuration is
      one that carries explicit authorization can be done from that config object.
      
      storage/cloud: make implementations register redaction params
      
      This changes the redaction of URIs from being done from a pre-defined map
      of sensitive params to a runtime populated map, in which implementations
      register their sensitive params during their registration during init.
      
      Release note: none.
      
      64695: deps: update golang.org/x/text to 0.3.6 r=knz a=rafiss
      
      fixes https://github.com/cockroachdb/cockroach/issues/63559
      
      Release note (sql change): Collated strings may now have a locale that
      is a language tag, followed by a `-u-` suffix, followed by anything
      else. For example, any locale with a prefix of `en-US-u-` is now
      considered valid.
      
      64698: opt: fix null reject rule cycle r=RaduBerinde a=RaduBerinde
      
      Fixing a corner case where we are requesting null rejection on a
      non-null Scan column. This causes a stack overflow.
      
      We also add an assertion that would have caught this (and returned an
      internal error instead of crashing the node).
      
      Fixes #64661.
      
      Release note (bug fix): fixed a stack overflow that can happen in some
      cornercases involving partial indexes with predicates containing `(x
      IS NOT NULL)`.
      
      64718: geos/BUILD.bzl: missing `-lm` r=rickystewart,rail a=knz
      
      Fixes #64713.
      
      The math functions are not included in libc in BSD.
      
      Release note: None
      Co-authored-by: default avatarDavid Taylor <[email protected]>
      Co-authored-by: default avatarRafi Shamim <[email protected]>
      Co-authored-by: default avatarRadu Berinde <[email protected]>
      Co-authored-by: default avatarRaphael 'kena' Poss <[email protected]>
      8aabb6db
    • Raphael 'kena' Poss's avatar
      pkg/cli: new flag `--log-config-file` to simplify log configuration · d584650d
      Raphael 'kena' Poss authored
      Release note (cli change): The new parameter `--log-config-file`
      simplifies the process of loading the logging configuration from a
      YAML file. Instead of passing the content of the file via
      e.g. `--log=$(cat file.yaml)`, it is now possible to pass the path to
      the file using `--log-config-file=file.yaml`.
      
      Note: each occurrence of `--log` and `--log-config-file` on the
      command line overrides the configuration set from previous
      occurrences.
      d584650d
    • craig[bot]'s avatar
      Merge #64490 · 43a079db
      craig[bot] authored
      64490: issues: improve random syntax test crash reports r=mgartner a=mgartner
      
      Crashes during random syntax tests now report more useful information to
      the issue including the error message, the SQL that produced the error,
      and the database schema.
      
      The report will be formatted as:
      
      > Random syntax error:
      >
      > ```
      >     rsg_test.go:755: Crash detected: server panic: pq: internal error: something bad
      > ```
      > Query:
      >
      > ```
      > 		SELECT
      > 			foo
      > 		FROM
      > 			bar
      > 		LIMIT
      > 			33:::INT8;
      > ```
      > Schema:
      >
      > ```
      >     rsg_test.go:575: To reproduce, use schema:
      >     rsg_test.go:577:
      >         	CREATE TABLE table1 (col1_0 BOOL);
      >         ;
      >     rsg_test.go:577:
      >
      >         CREATE TYPE greeting AS ENUM ('hello', 'howdy', 'hi', 'good day', 'morning');
      >         ;
      >     rsg_test.go:579:
      >     rsg_test.go:580: -- test log scope end --
      > test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSQLSmith460792454
      > --- FAIL: TestRandomSyntaxSQLSmith (300.69s)
      > ```
      
      Release note: None
      Co-authored-by: default avatarMarcus Gartner <[email protected]>
      43a079db
    • craig[bot]'s avatar
      Merge #64691 · 4ee093c9
      craig[bot] authored
      64691: sql: add memory accounting for the results of the subqueries r=yuzefovich a=yuzefovich
      
      After executing a subquery, the result is materialized into a single
      tuple (or row, or value). In some cases that tuple can be arbitrarily
      large. This commit adds after-the-fact memory accounting for the size of
      the resulting datum. The burden of clearing the memory account is put
      onto the caller of `PlanAndRunSubqueries` method.
      
      Fixes: #64464.
      
      Release note (bug fix): CockroachDB now should crash less often due to OOM
      conditions caused by the subqueries returning multiple rows of large
      size.
      Co-authored-by: default avatarYahor Yuzefovich <[email protected]>
      4ee093c9
    • Yahor Yuzefovich's avatar
      sql: add memory accounting for the results of the subqueries · 4131ad13
      Yahor Yuzefovich authored
      After executing a subquery, the result is materialized into a single
      tuple (or row, or value). In some cases that tuple can be arbitrarily
      large. This commit adds after-the-fact memory accounting for the size of
      the resulting datum. The burden of clearing the memory account is put
      onto the caller of `PlanAndRunSubqueries` method.
      
      Release note (bug fix): CockroachDB now should crash less often due to OOM
      conditions caused by the subqueries returning multiple rows of large
      size.
      4131ad13
    • Raphael 'kena' Poss's avatar
      geos/BUILD.bzl: missing `-lm` · 16ec1a3a
      Raphael 'kena' Poss authored
      The math functions are not included in libc in BSD.
      
      Release note: None
      16ec1a3a
    • craig[bot]'s avatar
      Merge #64704 · 1b5ff55e
      craig[bot] authored
      64704: roachtest: maybe fix `inconsistency` r=erikgrinaker a=tbg
      
      Closes #64602.
      
      Release note: None
      Co-authored-by: default avatarTobias Grieger <[email protected]>
      1b5ff55e
    • Tobias Grieger's avatar
      roachtest: own `quit-all-nodes` to Server · 952d3c55
      Tobias Grieger authored
      Release note: None
      952d3c55
    • Aditya Maru's avatar
      backupccl: add timeout to ExportRequest sent by a backup · ae48e202
      Aditya Maru authored
      Previously, there was no timeout around the ExportRequest that
      was sent as part of a backup. If the request returned certain
      kinds of unambiguous errors, dist sender would keep retrying the request
      thereby causing the backup to hang.
      
      This change adds an upper bound on the time it should take the sender
      to receive a response, thereby erroring out instead of mysteriously
      hanging. The upper bound is configurable via a cluster settings and is
      set to 5 mins by default, which is very conservative as these requests
      should be very fast under normal circumstances.
      
      Release note: None
      ae48e202
    • craig[bot]'s avatar
      Merge #64655 · fffb32a7
      craig[bot] authored
      64655: importccl: add `chunk_size` option to EXPORT CSV r=dt a=adityamaru
      
      This option allows the user to specify a target size per
      CSV file during an EXPORT. Once this target size is hit,
      we will upload the CSV file before processing anymore rows.
      
      Release note (sql change): Add `chunk_size` option to EXPORT
      CSV to control the target CSV file size. The default value will be set to 32MB.
      Co-authored-by: default avatarAditya Maru <[email protected]>
      fffb32a7
    • Tobias Grieger's avatar
      roachtest: maybe fix `inconsistency` · c1af82b9
      Tobias Grieger authored
      Closes #64602.
      
      Release note: None
      c1af82b9
    • craig[bot]'s avatar
      Merge #64136 · 782d3206
      craig[bot] authored
      64136: importccl: add more errors to transient retryable error list r=pbardea a=adityamaru
      
      This change adds a few more errors that were exposed when stressing
      a previously skipped test, that can occur when a worker node running
      the import is shutdown. These errors should not cause the job to fail
      but instead trigger a replan.
      
      I ran 500+ iterations under stress of TestImportWorkerFailure.
      
      Release note: None
      Co-authored-by: default avatarAditya Maru <[email protected]>
      782d3206
    • Aditya Maru's avatar
      importccl: add `chunk_size` option to EXPORT CSV · f9085d13
      Aditya Maru authored
      This option allows the user to specify a target size per
      CSV file during an EXPORT. Once this target size is hit,
      we will upload the CSV file before processing anymore rows.
      
      Release note (sql change): Add `chunk_size` option to EXPORT
      CSV to control the target CSV file size.
      f9085d13
    • craig[bot]'s avatar
      Merge #64703 · 3b8ec3ab
      craig[bot] authored
      64703: roachtest: fix multitenant-upgrade r=erikgrinaker a=tbg
      
      The test failed for two reasons:
      
      - it was hard-coding 20.2 in one place where it shouldn't have
      - it implicitly assumes that PredecessorVersion(21.2) is a 21.1 release
        that has #60730, but that will only be true in a month or so (when we
        have 21.1.1). Without #60730, tenant 11 in the test will have no
        cluster version persisted and so things don't work well. Work around
        this by setting the cluster version explicitly. This isn't a
        production concern since we are not going to upgrade tenants without
        a cluster version into 21.2; they will either have been created by a
        21.1.1+ host cluster (and thus have a cluster version) or been
        migrated from a 20.2 deployment into a 21.1.1+ deployment, which also
        entails giving them a cluster version.
      
      Fixes #64615.
      
      Release note: None
      Co-authored-by: default avatarTobias Grieger <[email protected]>
      3b8ec3ab