This project is mirrored from https://github.com/cockroachdb/cockroach. Updated .
  1. 18 Nov, 2019 2 commits
  2. 17 Nov, 2019 1 commit
    • craig[bot]'s avatar
      Merge #42440 · 90b33c07
      craig[bot] authored
      42440: roachprod: use premium SSDs for Azure r=pbardea a=pbardea
      
      We should be testing with Premium SSDs on Azure, as Standard SSDs do not
      have the performance characteristics we would expect of a machine that
      is actually running CockroachDB.
      
      Release note: None
      Co-authored-by: default avatarPaul Bardea <[email protected]>
      90b33c07
  3. 16 Nov, 2019 2 commits
    • craig[bot]'s avatar
      Merge #40900 · 09b39153
      craig[bot] authored
      40900: sql: bug fixes in column/sequence dependency relationship r=arulajmani a=arulajmani
      
      Currently, if a sequence is used by two columns of the same table, the
      dependency relation with the first column can be lost. This PR fixes
      that. It also ensures that the dependency relation is established only
      after ColumnIDs have been allocated.
      
      Fixes #40852
      
      Release note (bug fix): Tables containing two columns that reference
      the same sequence behave correctly.
      
      Release justification: Not part of release/pass linter.
      Co-authored-by: default avatarArul Ajmani <[email protected]>
      09b39153
    • Arul Ajmani's avatar
      sql: bug fixes in column/sequence dependency relationship · c216ed63
      Arul Ajmani authored
      Currently, if a sequence is used by two columns of the same table, the
      dependency relation with the first column can be lost. This PR fixes
      that. It also ensures that the dependency relation is established only
      after ColumnIDs have been allocated.
      
      Fixes #40852
      
      Release note (bug fix): Tables containing two columns that reference
      the same sequence behave correctly.
      
      Release justification: Not part of release/pass linter.
      c216ed63
  4. 15 Nov, 2019 5 commits
    • craig[bot]'s avatar
      Merge #42513 · e78a2b65
      craig[bot] authored
      42513: Add partition index diagram r=jseldess a=jseldess
      
      Release note: none
      Co-authored-by: default avatarJesse Seldess <[email protected]>
      e78a2b65
    • Jesse Seldess's avatar
      Add partition index diagram · b259d306
      Jesse Seldess authored
      Release note: none
      b259d306
    • craig[bot]'s avatar
      Merge #42480 · b272d63b
      craig[bot] authored
      42480: cli: change default --max-sql-memory to 25% r=knz a=dt
      
      This flag was changed to default to 128mb in #18040, as part of an
      effort to reduce memory usage of small nodes.
      
      Unlike the `cache` memory limit which was changed at the same time, the
      SQL memory limit is not passively consumed -- it is only used as-needed
      by active operations, so if being fully consumed, it is likely because
      it is actively being used, and setting it lower would actually degrade
      performance (by forcing disk-spilling, reducing concurrency, etc).
      
      The 128mb limit is thus generally acknowledged to be too small for any
      real production usage -- and we even logged warnings if it was used --
      but this means any unsuspecting user who does not alter it when firing
      up a node for some quick evaluation is also not getting representative
      behavior. Given that increasing this limit likely only increases actual
      memory usage of an active node -- where such usage is likely within
      the reasonable expectations of the user -- a higher default is likely
      preferable. Setting it based on a fraction of system memory elegantly
      allows adapting to different deployment environments automatically.
      
      Fixes #42437.
      
      Release note (general change): Changed the default value of --max-sql-memory limit from 128mb to 25% of system memory.
      Co-authored-by: default avatarDavid Taylor <[email protected]>
      b272d63b
    • craig[bot]'s avatar
      Merge #42277 · 87d7f5e4
      craig[bot] authored
      42277: storage: include SST keys in written-key QPS r=dt a=dt
      
      Application of normal batches increment the replica writes-per-second by
      the number of writes in the batch. This adds a similar increment to the
      special-case application of AddSSTable so its written keys are correctly
      included in the overall write-rate for the replica.
      
      Release note: none.
      Co-authored-by: default avatarDavid Taylor <[email protected]>
      87d7f5e4
    • craig[bot]'s avatar
      Merge #42412 · f96d984b
      craig[bot] authored
      42412: roachtest: print out output on an error with auto retry loop r=yuzefovich a=yuzefovich
      
      Previousy, 'repeatRunWithBuffer' (an automatic retry loop for executing
      a command on a roachprod cluster), if an error occurs, would only print
      out the error itself. In some cases this can be not very helpful to
      understanding why the command failed (for example, it could be just
      "exit status 1"). Now it will also print out the output.
      
      Release note: None
      Co-authored-by: default avatarYahor Yuzefovich <[email protected]>
      f96d984b
  5. 14 Nov, 2019 19 commits
    • craig[bot]'s avatar
      Merge #42005 #42444 · debcc221
      craig[bot] authored
      42005: backupccl: allow restoration of empty db r=pbardea a=pbardea
      
      Previously, RESTORE did not support restoring empty databases. However,
      there are use cases in which scripts perform the backup and restore
      process and they have run into issues since when they go to RESTORE all
      of their databases, if any are empty the script will fail.
      
      Since BACKUP supports the backing up of empty databases, it makes sense
      to allow RESTORE to restore those databases that were backed up.
      
      Fixes #41962.
      
      Release note (enterprise change): RESTORE now supports the restoration
      of empty databases.
      
      42444: roachprod: add ability to specify SSD count r=pbardea a=pbardea
      
      This adds a flag to roachprod to specify the number of SSDs. This is
      based off of #40801, but with a log message to notify the user upon
      changing of the flag if it sees an n2 machine type, if not enough SSDs
      were provided.
      
      Release note: None
      Co-authored-by: default avatarPaul Bardea <[email protected]>
      debcc221
    • craig[bot]'s avatar
      Merge #42453 · d70b0969
      craig[bot] authored
      42453: roachtest: show unexpected test results in github issue r=rafiss a=rafiss
      
      This should make it easier to glance at the issue to get an idea of
      what's wrong, rather than having to click through to the TeamCity
      results and navigate many log files.
      
      Release note: None
      Co-authored-by: default avatarRafi Shamim <[email protected]>
      d70b0969
    • craig[bot]'s avatar
      Merge #42484 · 988d5f25
      craig[bot] authored
      42484: storage/engine: disable usage of the Pebble archive cleaner r=petermattis a=petermattis
      
      Disable the usage of the Pebble archive cleaner which was used during a
      previous debugging effort and accidentally left enabled.
      
      Release note: None
      Co-authored-by: default avatarPeter Mattis <[email protected]>
      988d5f25
    • Paul Bardea's avatar
      roachprod: use premium SSDs for Azure · 71d24be9
      Paul Bardea authored
      We should be testing with Premium SSDs on Azure, as Standard SSDs do not
      have the performance characteristics we would expect of a machine that
      is actually running CockroachDB.
      
      Release note: None
      71d24be9
    • David Taylor's avatar
      cli: change default --max-sql-memory to 25% · d27d214e
      David Taylor authored
      This flag was changed to default to 128mb in #18040, as part of an
      effort to reduce memory usage of small nodes.
      
      Unlike the `cache` memory limit which was changed at the same time, the
      SQL memory limit is not passively consumed -- it is only used as-needed
      by active operations, so if being fully consumed, it is likely because
      it is actively being used, and setting it lower would actually degrade
      performance (by forcing disk-spilling, reducing concurrency, etc).
      
      The 128mb limit is thus generally acknowledged to be too small for any
      real production usage -- and we even logged warnings if it was used --
      but this means any unsuspecting user who does not alter it when firing
      up a node for some quick evaluation is also not getting representative
      behavior. Given that increasing this limit likely only increases actual
      memory usage of an active node -- where such usage is likely within
      the reasonable expectations of the user -- a higher default is likely
      preferable. Setting it based on a fraction of system memory elegantly
      allows adapting to different deployment environments automatically.
      
      Fixes #42437.
      
      Release note (general change): Changed the default value of --max-sql-memory limit from 128mb to 25% of system memory.
      d27d214e
    • Paul Bardea's avatar
      roachprod: add ability to specify SSD count · 3cca2923
      Paul Bardea authored
      This adds a flag to roachprod to specify the number of SSDs. This is
      based off of #40801, but with a log message to notify the user upon
      changing of the flag if it sees an n2 machine type, if not enough SSDs
      were provided.
      
      Release note: None
      3cca2923
    • craig[bot]'s avatar
      Merge #42487 · 5b83ce79
      craig[bot] authored
      42487: cmd/roachtest: fix sqlsmith tpch import r=mjibson a=mjibson
      
      Release note: None
      Co-authored-by: default avatarMatt Jibson <[email protected]>
      5b83ce79
    • Matt Jibson's avatar
      cmd/roachtest: fix sqlsmith tpch import · bee421c0
      Matt Jibson authored
      Release note: None
      bee421c0
    • craig[bot]'s avatar
      Merge #42447 · c412a31b
      craig[bot] authored
      42447: opt: support grouping by aliases r=RaduBerinde a=RaduBerinde
      
      There is some baggage left over from SQL92 which allowed grouping by
      select targets by their alias. We implement the same rules used by
      postgres, as explained in the `buildGroupingColumns` comment.
      
      Fixes #28059.
      
      Release note (sql change): It is now supported to specify selection
      target aliases as GROUP BY columns. Note that the FROM columns take
      precedence over the aliases, which are only used if there is no column
      with that name in the current scope.
      Co-authored-by: default avatarRadu Berinde <[email protected]>
      c412a31b
    • craig[bot]'s avatar
      Merge #42483 · 8dcc07f6
      craig[bot] authored
      42483: movr: update workload to use random values in columns rather than NULL r=rohany a=rohany
      
      The original pass at the movr workload filled up the non-indexed columns with
      null values rather than using randomly generated values. This PR fixes that.
      
      Fixes #42435.
      
      Release note (bug fix): The movr workload now populates table columns with randomly
      generated data instead of NULLs.
      Co-authored-by: default avatarRohan Yadav <[email protected]>
      8dcc07f6
    • Peter Mattis's avatar
      storage/engine: disable usage of the Pebble archive cleaner · c4e89144
      Peter Mattis authored
      Disable the usage of the Pebble archive cleaner which was used during a
      previous debugging effort and accidentally left enabled.
      
      Release note: None
      c4e89144
    • Rohan Yadav's avatar
      movr: update workload to use random values in columns rather than NULL · 7e28ca9c
      Rohan Yadav authored
      The original pass at the movr workload filled up the non-indexed columns with
      null values rather than using randomly generated values. This PR fixes that.
      
      Fixes #42435.
      
      Release note (bug fix): The movr workload now populates table columns with randomly
      generated data instead of NULLs.
      7e28ca9c
    • craig[bot]'s avatar
      Merge #42455 · d6d22fa6
      craig[bot] authored
      42455: cli: make OpenExistingStore respect COCKROACH_STORAGE_ENGINE r=petermattis a=petermattis
      
      Release note: None
      Co-authored-by: default avatarPeter Mattis <[email protected]>
      d6d22fa6
    • craig[bot]'s avatar
      Merge #42443 · 35dc85ab
      craig[bot] authored
      42443: storage/engine: support Pebble Compact r=petermattis a=hueypark
      
      Release note: None
      Co-authored-by: default avatarJaewan Park <[email protected]>
      35dc85ab
    • craig[bot]'s avatar
      Merge #42346 · d8e2c581
      craig[bot] authored
      42346: storage/engine: support StorageConfig.MustExist for pebble r=petermattis a=hueypark
      
      Release note: None
      Co-authored-by: default avatarJaewan Park <[email protected]>
      d8e2c581
    • craig[bot]'s avatar
      Merge #42474 · ff49a848
      craig[bot] authored
      42474: build: require patch level 10+ for go1.12 r=knz a=knz
      
      Fixes #42229
      
      Versions of Go before 1.12.10 were more lenient in the URL format they
      were accepting. This was changed in 1.12.10 and CockroachDB tests were
      updated accordingly, but these tests will now fail when ran with older
      1.12 versions.
      
      This patch enhances the version check to require a specific patch
      release.
      
      Release note (build change): Go version 1.12.10+ is now required to
      build CockroachDB successfully.
      Co-authored-by: default avatarRaphael 'kena' Poss <[email protected]>
      ff49a848
    • craig[bot]'s avatar
      Merge #42464 · 907540d0
      craig[bot] authored
      42464: cli, sql: allow usernames to start with digits r=knz a=georgebuckerfield
      
      Allows CockroachDB usernames to start with digits.
      
      Fixes #42155.
      
      e.g.
      ```
      [email protected]:26257/defaultdb> create user '1234';
      CREATE USER 1
      
      Time: 6.698583ms
      ```
      Co-authored-by: default avatargeorgebuckerfield <[email protected]>
      907540d0
    • georgebuckerfield's avatar
      cli, sql: allow usernames to start with digits · 30090542
      georgebuckerfield authored
      Previousy, client usernames could only be valid identifiers, with the
      first character allowed being either a letter of underscore.
      
      This was an arbitrary limitation; in fact, production systems
      where usernames are just numbers (all digits) are common.
      
      This patch changes the limitation to allow CockroachDB usernames to
      start with digits.
      
      Release note (general change): Client usernames can now be defined to
      start with a digit; in particular, all-digit usernames are now
      permitted.
      30090542
    • Raphael 'kena' Poss's avatar
      build: require patch level 10+ for go1.12 · f8e01e10
      Raphael 'kena' Poss authored
      Versions of Go before 1.12.10 were more lenient in the URL format they
      were accepting. This was changed in 1.12.10 and CockroachDB tests were
      updated accordingly, but these tests will now fail when ran with older
      1.12 versions.
      
      This patch enhances the version check to require a specific patch
      release.
      
      Release note (build change): Go version 1.12.10+ is now required to
      build CockroachDB successfully.
      f8e01e10
  6. 13 Nov, 2019 11 commits
    • craig[bot]'s avatar
      Merge #42231 · 8622fad0
      craig[bot] authored
      42231: sql: fix unchecked constraints with cascade loops r=RaduBerinde a=RaduBerinde
      
      #### sql: fix unchecked constraints with cascade loops
      
      The cascader uses a `CheckHelper` when updating rows in a table. Since
      we added optimizer support for mutations, we have two types of
      `CheckHelper` (the old "eval mode" and the new "input mode").
      
      The mutation code is passing the initial table's `CheckHelper` which
      is in "input mode". The cascader is using it in the "eval mode" which
      is a no-op. This causes CHECK constraints against the initial table to
      not be verified. This can happen with self-referencing cascades, or
      when there is a cascading loop.
      
      I plan to follow up with a cleanup that removes the two modes and
      separates the new "input mode" code completely. We can do this on
      master where we ripped out the heuristic planner.
      
      Fixes #42117.
      
      Release note (bug fix): Fixed a bug in scenarios where we have UPDATE
      cascades; and we are updating a table that has CHECK constraints; and
      the table is self-referencing or is involved in a reference cycle. In
      this case an UPDATE that cascades back in the original table was not
      validated with respect to the CHECK constraints.
      
      #### sql: allow check expressions with non-public columns in cascader
      
      When adding a column with a check expression, the schema change
      happens as follows:
       1. The backfill starts to initialize the new column; at this time, the
         check is not "active";
       2. The backfill completes; the check becomes "active" but the schema
          change is not complete and the column is still write-only;
       3. The check constraint is validated in another pass;
       4. The schema change is complete.
      
      It is correct to treat the columns as readable once the check becomes
      active.
      
      This change modifies the legacy `CheckHelper` (used by the cascader)
      to allow non-public columns in check expressions.
      
      Release note: None
      Co-authored-by: default avatarRadu Berinde <[email protected]>
      8622fad0
    • Jaewan Park's avatar
      storage/engine: support Pebble Compact · 160b9201
      Jaewan Park authored
      Release note: None
      160b9201
    • craig[bot]'s avatar
      Merge #42465 · 84eddf3c
      craig[bot] authored
      42465: cmd/roachtest: add TPCC for sqlsmith r=mjibson a=mjibson
      
      Also fix TPCH while here. Tables must be in the defaultdb database.
      
      Release note: None
      Co-authored-by: default avatarMatt Jibson <[email protected]>
      84eddf3c
    • Matt Jibson's avatar
      cmd/roachtest: add TPCC for sqlsmith · 97620298
      Matt Jibson authored
      Also fix TPCH while here. Tables must be in the defaultdb database.
      
      Release note: None
      97620298
    • craig[bot]'s avatar
      Merge #42232 · b2b33431
      craig[bot] authored
      42232: colexec: plumb through Allocator skeleton r=yuzefovich a=yuzefovich
      
      This commit adds the plumbing of newly added Allocator which is
      responsible for doling out batches to operators within a fixed budget.
      The budgeting logic doesn't exist yet, but this commit adds the new type
      and plumbs it through operators which use a variable amount of memory.
      Next step is to actually measure the memory required when NewBatch or
      Append is called and track that against a budget.
      
      A linter rule is added that prohibits usage of coldata.NewMem* methods
      within sql/colexec and sql/colflow packages.
      
      Based on: #38394.
      Fixes: #38056.
      
      Release note: None
      Co-authored-by: default avatarYahor Yuzefovich <[email protected]>
      b2b33431
    • craig[bot]'s avatar
      Merge #42448 #42456 · 8418619c
      craig[bot] authored
      42448: sql/mutations: add stats injector + refactor r=mjibson a=mjibson
      
      
      
      42456: builtins: fix potential panic in crdb_internal.encode_key r=rohany a=rohany
      
      crdb_internal.encode_key was indexing into a table descriptor's
      indexes slice using index ID's, when it should have been using
      `FindIndexByID`.
      
      Release note (bug fix): For tables with dropped indexes, the SHOW RANGE FOR ROW command sometimes returned incorrect
      results or an error. Fixed the underlying issue in the crdb_internal.encode_key built-in.
      Co-authored-by: default avatarMatt Jibson <[email protected]>
      Co-authored-by: default avatarRohan Yadav <[email protected]>
      8418619c
    • Matt Jibson's avatar
      sql/mutations: add stats mutator · 62f3fd95
      Matt Jibson authored
      This mutator injects random stats into tables. It's already found a bug
      in the logic tests, and so is unused there until it is fixed.
      62f3fd95
    • craig[bot]'s avatar
      Merge #42330 · 2aac1975
      craig[bot] authored
      42330: importccl: Modify readFiles() input converter method to take in a resume position indicator r=miretskiy a=miretskiy
      
      Update inputConverter's readFile method to take in an additional
      resume position parameter in order for the converter implementations
      to be able to support import resume functionality.
      
      Release note: None
      Co-authored-by: default avatarYevgeniy Miretskiy <[email protected]>
      2aac1975
    • Yahor Yuzefovich's avatar
      colexec: plumb through Allocator skeleton · d4962b15
      Yahor Yuzefovich authored
      This commit adds the plumbing of newly added Allocator which is
      responsible for doling out batches to operators within a fixed budget.
      The budgeting logic doesn't exist yet, but this commit adds the new type
      and plumbs it through operators which use a variable amount of memory.
      Next step is to actually measure the memory required when NewBatch or
      Append is called and track that against a budget.
      
      A linter rule is added that prohibits usage of coldata.NewMem* methods
      within sql/colexec and sql/colflow packages.
      
      Release note: None
      d4962b15
    • Rohan Yadav's avatar
      builtins: fix potential panic in crdb_internal.encode_key · 1539f953
      Rohan Yadav authored
      crdb_internal.encode_key was indexing into a table descriptor's
      indexes slice using index ID's, when it should have been using
      `FindIndexByID`.
      
      Release note (bug fix): For tables with dropped indexes, the SHOW RANGE FOR ROW command sometimes returned incorrect
      results or an error. Fixed the underlying issue in the crdb_internal.encode_key built-in.
      1539f953
    • craig[bot]'s avatar
      Merge #42457 · b5d4ee3e
      craig[bot] authored
      42457: builtins: add avg for interval r=otan a=otan
      
      Resolves https://github.com/cockroachdb/cockroach/issues/42449
      Resolves https://github.com/cockroachdb/cockroach-django/issues/72
      
      This PR adds the ability to average intervals. Comes with a few extra
      tests for aggregation functions over intervals.
      
      Release note (sql change): Adds the ability to run `avg` over intervals.
      Co-authored-by: default avatarOliver Tan <[email protected]>
      b5d4ee3e