10 Commits

Author SHA1 Message Date
Greg Landrum
b693c5d4df Revert "Fixes #8644 (#8768)" (#8787)
This reverts commit a47a5d91aa.
2025-09-16 16:28:01 +02:00
Ricardo Rodriguez
a47a5d91aa Fixes #8644 (#8768)
* potential satisfactory solution

* fix typo

* update testMultithreadedMolSupplier.py

* revert first patch

* better fix

* add test
2025-09-16 05:24:45 +02:00
Ricardo Rodriguez
397d19e676 Mem fixes (#8640)
* fix leaking mol in rdEnumerateStereoisomers.cpp

* clang-format Code/GraphMol/FileParsers/MultithreadedMolSupplier.cpp

* fix leaking mols in MultithreadedMolSupplier output queue
2025-07-21 05:57:50 +02:00
Brian Kelley
a6890baf0f Fixes #8284 (#8556)
* Adds a df_forceStop to stop readers and writers, clears out queues on destructor

* Propery implement close function, requires protected closeStreams

* changes from greg's version

* close() needs to be called in the derived destructors

* Close the writers before the reader to avoid deadlock

* Don't process trailing new lines

* Don't accept pushes if the queue is done

* Add mutex protecting d_threadCounter, remove unneeded forceStop checks

* Update Code/GraphMol/FileParsers/MultithreadedMolSupplier.cpp

Co-authored-by: Greg Landrum <greg.landrum@gmail.com>

* Add comment for the d_threadCounterMutex unlock

---------

Co-authored-by: = <=>
Co-authored-by: Greg Landrum <greg.landrum@gmail.com>
2025-06-06 14:09:57 +02:00
Ivan Tubert-Brohman
d8bc5d61f8 Catch exceptions in MultithreadedMolSupplier callbacks (#7810)
* Catch exceptions in MultithreadedMolSupplier callbacks

* In next(), simply ignore any exceptions from nextCallback.
* In reader(), if readCallback throws, log a warning and proceed using
  the unmodified record.
* (The writer() was already handling exceptions from writeCallback.)

* Remove unused parameter names

Hopefully this will placate the warning/error settings used by the Linux
build.
2024-11-19 17:22:25 +01:00
Ricardo Rodriguez
0164ac7fae Fix some more mem errors in 2024.09.1 (#7899)
* fixes

* do not leak MolCatalogParams

* do not leak points on align failures

* give python ownership of pointers returned in getFingerprintsHelper

* clean up ScaffoldNetwork ptr if createNetworkHelper fails

* manage FF ptrs during construction

* wire in ownsBondInvGenerator in getMorganGenerator

* manage weights in rdMolAlign CalcRMS

* fix ownership of matches list/tuple in generateRmsdTransMatchPyTuple

* manage stream in createForwardSupplier during construction

* drop redundant Point3D allocations in GetUSRDistributionsFromPoints

* fix signed comparison mismatch
2024-10-10 16:08:50 +02:00
Greg Landrum
9dc263b7b4 Add callbacks to the multithreaded mol suppliers (#7763)
* MultithreadedMolSuppliers can now be destroyed without being used.

This was previously not possible

* add callbacks to the multithreaded readers

* document the new functions

* switch to storing the queues in unique_ptrs

* only do those tests when in MT mode
2024-08-29 15:53:47 +02:00
Greg Landrum
3a27ca2910 V2 API for the MolSuppliers (#7168)
* backup

* backup

* backup

* new testing file

* reorder things in the SMILES and SMARTS parser
parameters to allow easier struct initialization

* add v2 of SmilesMolSupplier

v1 is not yet there

* add TDTMolSupplier

add some very basic tests for v1 suppliers

* more progress

* all tests passing

I think all the suppliers are now moved over

* check for maeparser before building the tests

* get SWIG builds working

* changes in response to review
2024-02-29 06:46:56 +01:00
Greg Landrum
0147cd8201 Fixes #5210 (#5408)
* revert duplicate chunk in release notes

* replace deprecated ifdefs
This one gets rid of USE_BUILTIN_POPCNT and RDK_THREADSAFE_SS
use RDK_OPTIMIZE_POPCNT or RDK_BUILD_THREADSAFE_SSS instead

* get rid of BUILD_COORDGEN_SUPPORT from ROMol.i

* fix a stupid typo

* update release notes
2022-07-11 11:20:03 +02:00
shrey183
8ea1ac6112 [GSoC-2020] Generalized and Multithreaded File Reader (#3363)
* fixed issue #2965

* added test case for issue #2965

* fixed formatting and added comment.

* update

* General Reader files

* removed dependency on boost filesystems

* removed class

* clang-format

* added-comments

* further-cleanup

* added clang-formatting

* braces-for-if-else

* changed error messages, added option for windows file path

* fixed getFileName function

* cleanup

* option for filename without path

* further-cleanup

* added tests for determineFileFormat

* cleanup, const arguments for validate function

* init

* cleanup

* cleanup

* clang-format does not work for CMake

* added RDK_TEST_MULTITHREADED option

* add-flag

* cleanup

* Delete ConcurrentQueue.h

This PR deals with the Generalized File Reader.

* Delete testConcurrentQueue.cpp

This PR deals with the Generalized File Reader.

* no change

* concurrent queue

* print values

* Single Producer Multiple Consumer works

* cleanup

* Producer Consumer Example

* update queue methods and tests

* cleanup

* test

* fixed tests

* cleanup, updated tests

* Delete ProducerConsumer.h

* Delete testProducerConsumer.cpp

* cleanup

* futher cleanup

* changes based on feedback

* make queue non copyable

* psuedocode

* possible implementation

* untested implementation

* change class to typename

* basic-setup

* need to fix segfault

* need to fix blocking

* need to fix blocking

* need to fix blocking

* fix indentation

* one possibility

* without lambda function

* possible fix with some test cases

* performance tests

* added support for record id and item text

* cleanup

* cleanup

* fixed memory leak and added methods with tests for getting last id and item text

* cleanup

* added more test cases with different smi files

* cleanup

* SD mol supplier

* modified the parsing for SDMolSupplier

* cleanup

* cleanup

* new file for testing

* added support for reading molecule properties with tests

* thread-safe logging and exception handling

* cleanup

* without thread safe logging

* cleanup

* cleanup, modified MultithreadedSmilesMolSupplier

* cleanup, made reader and writer functions private

* move O2.sdf

* basic python wrapper with tests

* cleanup, added new methods for python wrappers

* made changes suggested by Andrew

* file and compression formats are case-insensitive

* cannot open files with gzstream

* cleanup

* possible fix for opening compressed streams (SMILES)

* removed seekg() and tellg() methods from multithreadeded suppliers

* cleanup

* test cases for python wrappers

* some wrapper cleanup

* cleanup, removed unused functions

* update the MT tests so that they actually do some work
also includes some cleanup here

* cleanup

* remove iterator_next header include

* added support for multithreaded readers

* use getNumThreadsToUse for multithreaded suppliers

* fixed documentation for multithreaded python wrappers

* commented performance test

* first draft of final evaluation report

* removed inline variables

* first draft getting started in python

* fixed typos in getting started in python

* fixed typos

* fix documentation tests

* fixed documentation tests

* added links to important files and PR

* added perfomance results

* first version of wrappers with compressed streams

* getting rid of streambuf stream method

* modified General File Reader

* make this work when building in non-threads mode

* rename a test

* rename a function in the python API

* rearrange the python test a bit

* disable the stream-based constructors in Python

* mark the multithreaded classes as experimental

Co-authored-by: greg landrum <greg.landrum@gmail.com>
2020-10-09 04:31:05 +02:00