cleanX.image_work package

Submodules

• cleanX.image_work.image_functions module
  • Cv2Error
  • cv2_imread()
  • crop_np()
  • crop_np_white()
  • find_outliers_sum_of_pixels_across_set()
  • hist_sum_of_pixels_across_set()
  • crop()
  • subtle_sharpie_enhance()
  • harsh_sharpie_enhance()
  • salting()
  • simple_rotate_no_pil()
  • blur_out_edges()
  • multi_rotation_augmentation_no_pill()
  • show_major_lines_on_image()
  • find_big_lines()
  • separate_image_averager()
  • dimensions_to_df()
  • dimensions_to_histo()
  • proportions_ht_wt_to_histo()
  • find_very_hazy()
  • find_by_sample_upper()
  • find_sample_upper_greater_than_lower()
  • find_outliers_by_total_mean()
  • find_outliers_by_mean_to_df()
  • create_matrix()
  • find_tiny_image_differences()
  • tesseract_specific()
  • find_suspect_text()
  • find_suspect_text_by_length()
  • histogram_difference_for_inverts()
  • inverts_by_sum_compare()
  • histogram_difference_for_inverts_todf()
  • find_duplicated_images()
  • find_duplicated_images_todf()
  • show_images_in_df()
  • dataframe_up_my_pics()
  • Rotator
    • Rotator.RotationIterator
      • Rotator.RotationIterator.__init__()
      • Rotator.RotationIterator.__iter__()
    • Rotator.__init__()
    • Rotator.iter()
  • simple_spinning_template()
  • make_contour_image()
  • avg_image_maker()
  • set_image_variability()
  • avg_image_maker_by_label()
  • zero_to_twofivefive_simplest_norming()
  • rescale_range_from_histogram_low_end()
  • make_histo_scaled_folder()
  • give_size_count_df()
  • give_size_counted_dfs()
  • image_quality_by_size()
  • find_close_images()
  • show_close_images()
  • image_to_histo()
  • black_end_ratio()
  • outline_segment_by_otsu()
  • binarize_by_otsu()
  • column_sum_folder()
  • blind_quality_matrix()
  • fourier_transf()
  • pad_to_size()
  • cut_to_size()
  • cut_or_pad()
  • rotated_with_max_clean_area()
  • noise_sum_cv()
  • noise_sum_median_blur()
  • noise_sum_gaussian()
  • noise_sum_bilateral()
  • noise_sum_bilateralLO()
  • noise_sum_5k()
  • noise_sum_7k()
  • blind_noise_matrix()
  • segmented_blind_noise_matrix()
  • make_inverted()
  • cv2_phash_for_dupes()
• cleanX.image_work.journaling_pipeline module
  • JournalingPipeline
    • JournalingPipeline.__init__()
    • JournalingPipeline.restore()
    • JournalingPipeline.process()
    • JournalingPipeline.process_batch_agg()
    • JournalingPipeline.process_batch_parallel()
    • JournalingPipeline.process_step()
• cleanX.image_work.pipeline module
  • MultiSource
    • MultiSource.__init__()
    • MultiSource.__iter__()
  • GlobSource
    • GlobSource.__init__()
    • GlobSource.__iter__()
  • DirectorySource
    • DirectorySource.__init__()
    • DirectorySource.__iter__()
  • PipelineError
  • Pipeline
    • Pipeline.__init__()
    • Pipeline.process()
    • Pipeline.process_step()
    • Pipeline.process_batch_agg()
    • Pipeline.process_batch_parallel()
• cleanX.image_work.steps module
  • get_known_steps()
  • RegisteredStep
    • RegisteredStep.__init__()
    • RegisteredStep.mro()
  • Step
    • Step.__init__()
    • Step.apply()
    • Step.read()
    • Step.write()
    • Step.begin_transaction()
    • Step.commit_transaction()
    • Step.__reduce__()
    • Step.to_json()
    • Step.from_cmd_args()
  • Aggregate
    • Aggregate.__init__()
    • Aggregate.begin_transaction()
    • Aggregate.commit_transaction()
    • Aggregate.pre()
    • Aggregate.post()
    • Aggregate.agg()
    • Aggregate.aggregate()
    • Aggregate.__reduce__()
    • Aggregate.apply()
    • Aggregate.from_cmd_args()
    • Aggregate.read()
    • Aggregate.to_json()
    • Aggregate.write()
  • Mean
    • Mean.agg()
    • Mean.post()
    • Mean.__reduce__()
    • Mean.__init__()
    • Mean.aggregate()
    • Mean.apply()
    • Mean.begin_transaction()
    • Mean.commit_transaction()
    • Mean.from_cmd_args()
    • Mean.pre()
    • Mean.read()
    • Mean.to_json()
    • Mean.write()
  • GroupHistoHtWt
    • GroupHistoHtWt.__init__()
    • GroupHistoHtWt.agg()
    • GroupHistoHtWt.post()
    • GroupHistoHtWt.__reduce__()
    • GroupHistoHtWt.aggregate()
    • GroupHistoHtWt.apply()
    • GroupHistoHtWt.begin_transaction()
    • GroupHistoHtWt.commit_transaction()
    • GroupHistoHtWt.from_cmd_args()
    • GroupHistoHtWt.pre()
    • GroupHistoHtWt.read()
    • GroupHistoHtWt.to_json()
    • GroupHistoHtWt.write()
  • GroupHistoPorportion
    • GroupHistoPorportion.__init__()
    • GroupHistoPorportion.agg()
    • GroupHistoPorportion.post()
    • GroupHistoPorportion.__reduce__()
    • GroupHistoPorportion.aggregate()
    • GroupHistoPorportion.apply()
    • GroupHistoPorportion.begin_transaction()
    • GroupHistoPorportion.commit_transaction()
    • GroupHistoPorportion.from_cmd_args()
    • GroupHistoPorportion.pre()
    • GroupHistoPorportion.read()
    • GroupHistoPorportion.to_json()
    • GroupHistoPorportion.write()
  • Acquire
    • Acquire.read()
    • Acquire.__init__()
    • Acquire.__reduce__()
    • Acquire.apply()
    • Acquire.begin_transaction()
    • Acquire.commit_transaction()
    • Acquire.from_cmd_args()
    • Acquire.to_json()
    • Acquire.write()
  • Save
    • Save.__init__()
    • Save.write()
    • Save.__reduce__()
    • Save.apply()
    • Save.begin_transaction()
    • Save.commit_transaction()
    • Save.from_cmd_args()
    • Save.read()
    • Save.to_json()
  • FourierTransf
    • FourierTransf.apply()
    • FourierTransf.__init__()
    • FourierTransf.__reduce__()
    • FourierTransf.begin_transaction()
    • FourierTransf.commit_transaction()
    • FourierTransf.from_cmd_args()
    • FourierTransf.read()
    • FourierTransf.to_json()
    • FourierTransf.write()
  • ContourImage
    • ContourImage.apply()
    • ContourImage.__init__()
    • ContourImage.__reduce__()
    • ContourImage.begin_transaction()
    • ContourImage.commit_transaction()
    • ContourImage.from_cmd_args()
    • ContourImage.read()
    • ContourImage.to_json()
    • ContourImage.write()
  • ProjectionHorizoVert
    • ProjectionHorizoVert.apply()
    • ProjectionHorizoVert.__init__()
    • ProjectionHorizoVert.__reduce__()
    • ProjectionHorizoVert.begin_transaction()
    • ProjectionHorizoVert.commit_transaction()
    • ProjectionHorizoVert.from_cmd_args()
    • ProjectionHorizoVert.read()
    • ProjectionHorizoVert.to_json()
    • ProjectionHorizoVert.write()
  • BlackEdgeCrop
    • BlackEdgeCrop.apply()
    • BlackEdgeCrop.__init__()
    • BlackEdgeCrop.__reduce__()
    • BlackEdgeCrop.begin_transaction()
    • BlackEdgeCrop.commit_transaction()
    • BlackEdgeCrop.from_cmd_args()
    • BlackEdgeCrop.read()
    • BlackEdgeCrop.to_json()
    • BlackEdgeCrop.write()
  • WhiteEdgeCrop
    • WhiteEdgeCrop.apply()
    • WhiteEdgeCrop.__init__()
    • WhiteEdgeCrop.__reduce__()
    • WhiteEdgeCrop.begin_transaction()
    • WhiteEdgeCrop.commit_transaction()
    • WhiteEdgeCrop.from_cmd_args()
    • WhiteEdgeCrop.read()
    • WhiteEdgeCrop.to_json()
    • WhiteEdgeCrop.write()
  • Sharpie
    • Sharpie.__init__()
    • Sharpie.apply()
    • Sharpie.__reduce__()
    • Sharpie.begin_transaction()
    • Sharpie.commit_transaction()
    • Sharpie.from_cmd_args()
    • Sharpie.read()
    • Sharpie.to_json()
    • Sharpie.write()
  • BlurEdges
    • BlurEdges.__init__()
    • BlurEdges.apply()
    • BlurEdges.__reduce__()
    • BlurEdges.begin_transaction()
    • BlurEdges.commit_transaction()
    • BlurEdges.from_cmd_args()
    • BlurEdges.read()
    • BlurEdges.to_json()
    • BlurEdges.write()
  • CleanRotate
    • CleanRotate.__init__()
    • CleanRotate.apply()
    • CleanRotate.__reduce__()
    • CleanRotate.begin_transaction()
    • CleanRotate.commit_transaction()
    • CleanRotate.from_cmd_args()
    • CleanRotate.read()
    • CleanRotate.to_json()
    • CleanRotate.write()
  • Normalize
    • Normalize.apply()
    • Normalize.__init__()
    • Normalize.__reduce__()
    • Normalize.begin_transaction()
    • Normalize.commit_transaction()
    • Normalize.from_cmd_args()
    • Normalize.read()
    • Normalize.to_json()
    • Normalize.write()
  • HistogramNormalize
    • HistogramNormalize.__init__()
    • HistogramNormalize.apply()
    • HistogramNormalize.__reduce__()
    • HistogramNormalize.begin_transaction()
    • HistogramNormalize.commit_transaction()
    • HistogramNormalize.from_cmd_args()
    • HistogramNormalize.read()
    • HistogramNormalize.to_json()
    • HistogramNormalize.write()
  • InvertImages
    • InvertImages.apply()
    • InvertImages.__init__()
    • InvertImages.__reduce__()
    • InvertImages.begin_transaction()
    • InvertImages.commit_transaction()
    • InvertImages.from_cmd_args()
    • InvertImages.read()
    • InvertImages.to_json()
    • InvertImages.write()
  • OtsuBinarize
    • OtsuBinarize.__init__()
    • OtsuBinarize.apply()
    • OtsuBinarize.__reduce__()
    • OtsuBinarize.begin_transaction()
    • OtsuBinarize.commit_transaction()
    • OtsuBinarize.from_cmd_args()
    • OtsuBinarize.read()
    • OtsuBinarize.to_json()
    • OtsuBinarize.write()
  • OtsuLines
    • OtsuLines.__init__()
    • OtsuLines.apply()
    • OtsuLines.__reduce__()
    • OtsuLines.begin_transaction()
    • OtsuLines.commit_transaction()
    • OtsuLines.from_cmd_args()
    • OtsuLines.read()
    • OtsuLines.to_json()
    • OtsuLines.write()
  • Projection
    • Projection.apply()
    • Projection.__init__()
    • Projection.__reduce__()
    • Projection.begin_transaction()
    • Projection.commit_transaction()
    • Projection.from_cmd_args()
    • Projection.read()
    • Projection.to_json()
    • Projection.write()

Module contents

cleanX.image_work.create_pipeline(steps, batch_size=None, journal=None, keep_journal=False)

Create a pipeline that will execute the steps. If journal is not false, create a journaling pipeline, that can be pick up from the failed step.

Parameters:

• steps (Sequence[Step]) – A sequence of Step to be executed in this pipeline.

• batch_size (int) – Controls how many steps are processed concurrently.

• journal (Union[bool, str]) – If True is passed, the pipeline code will use a preconfigured directory to store the journal. Otherwise, this must be the path to the directory to store the journal database.

• keep_journal (bool) – Controls whether the journal is kept after successful completion of the pipeline.

Returns:

a Pipeline object or one of its descendants.

Return type:

Pipeline

cleanX.image_work.restore_pipeline(journal_dir, skip=0, **overrides)

Restores previously interrupted pipeline. The pipeline should have been created with journal set. If the creating code didn’t specify the directory to keep the journal, it may be obtained in this way:

p = create_pipeline(steps=(...), journal=True)
journal_dir = p.journal_dir
# After pipeline failed
p = restore_pipeline(journal_dir)

Parameters:

• journal_dir (Suitable for os.path.join()) – The directory containing journal database to restore from.

• skip – Skip this many steps before attempting to resume the pipeline. This is useful if you know that the step that failed will fail again, but you want to execute the rest of the steps in the pipeline.

• **overrides – Arguments to pass to the created pipeline instance that will override those restored from the journal.

Returns:

Fresh JournalingPipeline object fast-forwarded to the last executed step + skip.

Return type:

JournalingPipeline