@Test void typicalRunPartitioned() throws TException, IOException { initialiseApp(); setPartitionedTable("partition1_"); application.run(args); verify(schedulerService, times(2)).scheduleForHousekeeping(housekeepingPath.capture()); List<HousekeepingPath> scheduledPaths = housekeepingPath.getAllValues(); String file1Path = "file:" + partition1InSnapshot0Dir.toString(); String file2Path = "file:" + partition2InSnapshot0Dir.toString(); Set<String> paths = Set.of(scheduledPaths.get(0).getPath(), scheduledPaths.get(1).getPath()); assertThat(paths).contains(file1Path); assertThat(paths).contains(file2Path); assertThat(assertBytesLogged(14)).isTrue(); for (HousekeepingPath path : scheduledPaths) { assertThat(path.getDatabaseName()).isEqualTo(databaseName); assertThat(path.getTableName()).isEqualTo(tableName); assertThat(path.getHousekeepingStatus()).isEqualTo(SCHEDULED); assertThat(path.getCleanupDelay().toDays()).isEqualTo(1L); assertThat(path.getLifecycleType()).isEqualTo(UNREFERENCED.name()); } }
@Override public void run(ApplicationArguments args) { log.warn("Do not run this tool at the same time as replicating to the target table!"); if (isDryRun) { log.warn("Dry-run only!"); } try { metastore = clientSupplier.get(); housekeepingPaths = fetchHousekeepingPaths(beekeeperRepository); vacuumTable(databaseName, tableName); } catch (URISyntaxException | TException | IOException e) { throw new RuntimeException(e); } finally { if (metastore != null) { metastore.close(); } } }
BeekeeperVacuumToolApplication implements ApplicationRunner { @Override public void run(ApplicationArguments args) { log.warn("Do not run this tool at the same time as replicating to the target table!"); if (isDryRun) { log.warn("Dry-run only!"); } try { metastore = clientSupplier.get(); housekeepingPaths = fetchHousekeepingPaths(beekeeperRepository); vacuumTable(databaseName, tableName); } catch (URISyntaxException | TException | IOException e) { throw new RuntimeException(e); } finally { if (metastore != null) { metastore.close(); } } } }
BeekeeperVacuumToolApplication implements ApplicationRunner { @Override public void run(ApplicationArguments args) { log.warn("Do not run this tool at the same time as replicating to the target table!"); if (isDryRun) { log.warn("Dry-run only!"); } try { metastore = clientSupplier.get(); housekeepingPaths = fetchHousekeepingPaths(beekeeperRepository); vacuumTable(databaseName, tableName); } catch (URISyntaxException | TException | IOException e) { throw new RuntimeException(e); } finally { if (metastore != null) { metastore.close(); } } } @Autowired  BeekeeperVacuumToolApplication(
      Supplier<CloseableMetaStoreClient> clientSupplier,
      SchedulerService schedulerService,
      BeekeeperRepository beekeeperRepository,
      HiveConf conf,
      @Value("${database}") String databaseName,
      @Value("${table}") String tableName,
      @Value("${default-cleanup-delay:P3D}") String cleanupDelay,
      @Value("${dry-run:false}") boolean isDryRun,
      @Value("${partition-batch-size:1000}") short batchSize); }
BeekeeperVacuumToolApplication implements ApplicationRunner { @Override public void run(ApplicationArguments args) { log.warn("Do not run this tool at the same time as replicating to the target table!"); if (isDryRun) { log.warn("Dry-run only!"); } try { metastore = clientSupplier.get(); housekeepingPaths = fetchHousekeepingPaths(beekeeperRepository); vacuumTable(databaseName, tableName); } catch (URISyntaxException | TException | IOException e) { throw new RuntimeException(e); } finally { if (metastore != null) { metastore.close(); } } } @Autowired  BeekeeperVacuumToolApplication(
      Supplier<CloseableMetaStoreClient> clientSupplier,
      SchedulerService schedulerService,
      BeekeeperRepository beekeeperRepository,
      HiveConf conf,
      @Value("${database}") String databaseName,
      @Value("${table}") String tableName,
      @Value("${default-cleanup-delay:P3D}") String cleanupDelay,
      @Value("${dry-run:false}") boolean isDryRun,
      @Value("${partition-batch-size:1000}") short batchSize); @Override void run(ApplicationArguments args); }
BeekeeperVacuumToolApplication implements ApplicationRunner { @Override public void run(ApplicationArguments args) { log.warn("Do not run this tool at the same time as replicating to the target table!"); if (isDryRun) { log.warn("Dry-run only!"); } try { metastore = clientSupplier.get(); housekeepingPaths = fetchHousekeepingPaths(beekeeperRepository); vacuumTable(databaseName, tableName); } catch (URISyntaxException | TException | IOException e) { throw new RuntimeException(e); } finally { if (metastore != null) { metastore.close(); } } } @Autowired  BeekeeperVacuumToolApplication(
      Supplier<CloseableMetaStoreClient> clientSupplier,
      SchedulerService schedulerService,
      BeekeeperRepository beekeeperRepository,
      HiveConf conf,
      @Value("${database}") String databaseName,
      @Value("${table}") String tableName,
      @Value("${default-cleanup-delay:P3D}") String cleanupDelay,
      @Value("${dry-run:false}") boolean isDryRun,
      @Value("${partition-batch-size:1000}") short batchSize); @Override void run(ApplicationArguments args);  }