Forum Replies Created
-
Posts
-
For the first point, it will be started in one Thread and Working with it’s own Input and Output Queues.
For the second point, what exactly do you mean by that? The waitDequeueNotification Function call will return a pointer(in the complete program it returns a std::unique_ptr) which i then forward to the output queue.Maybe i simplified the code to much :D
I added the whole class including the Test written with GoogleTest: https://godbolt.org/z/hEaT6snaM
It is quite large because i put it all in one place.
What i want to archive with this class is a kind of a Load Balancer.
The General Idea is that if there are a lot of Messages in the Output Queue i want to increase the size of the workers. I do this if three times in a row the the Outputqueue Size is greater then the given Limit.
I also reduce the Workers if the Output Queue Size is five times below the given Limit.
The Problem in the tests are that sometimes the Tests failes because the wait time seems to be to low. Such Problems seems to be hard to debug because if i add some output printing the test never failes because as far as i can think of the program needs some time to do the output.
In general the testcase RunDecrementsDecreasingCounterIfQueueSizeAboveLimit failes sometimes but also others are sometimes failing.
One idea that also came to my mind was to Mock away the Queue for that i already developed the MessageQueue Interface which is kind of an Adapter to the underlying Poco::NotificationQueue class.I tried that code in the compiler explorer but did not get an segfault.
By private part i mean that they are not exported so they are not visible to the users of the library and therefore not visible for the test. If i want to create a test executable (for example using gmock or catch2) i have to link that executable against the shared library. The Test then did not have access to those noexported Functions. I can of course include those headerfiles but the linkage will fail because it is not able to find the function definitions in the lib file.
On solution of course is to create the library as a static library and then link the tests against those but i’m not sure if this is a sutaible solution
I have 2 threads that act as producer.
In general they both do the same thing, collecting files and sending the content of those file to another service who acts as the consumer.
The problem here is how to ensure that a file is not send twice.
This could happen because when the application starts i first start the DirectoryWatcher Thread. This Thread just receives Filesystem events. It then reads the newly added File and sends the content to another service.
After i started the DirectoryWatcher Thread i start the DirectoryReader Thread. This Thread iterates over the same directory. I need this thread because in that folder the files are added in a regular base. If the application stops/crashes new files are still added to that folder. To also collect those files i added the Directory Reader. It iterates over the directory collects the files and sends the content to the same service as the DirectoryWatcher.
The Problem is now during the Iteration over the Directory from the DirectoryReader new files could be added to that folder. THese files are collected by the DirectoryWatcher but it could also be that the DirectoryReader sees those files and they could be processed twice.
Thats why i thought about that the DirectoryWatcher Thread sends out Notifications to the DirectoryReader containing the filename of the currently added file. The DirectoryReader then checks if on of the files it iterates over are already collected by the DirectoryWatcher and ignores them. When the DirectoryReader finished its work it send out a notification to the DirectoryWatcher. The DirectoryWatcher then stops sending those Notifications because after the DirectoryReader finished iterating over the Directory it just exit and did not exists anymore.
Hope this clearifys the problem. Maybe i can try to explain it on the next Q&A.
Hello Rainer,
i was thinking of implementing a kind of Pipes-and-Filters Architecture.
The basic idea is that the InputSource collects Data from a Folder so whenever a new file is added to that folder the InputSource receives a Notification from the Filesystem. It then loads the File creates a Message Object and notifies the Balancer with that Message. The Balancer then puts the Message into a Queue where the Workers are listening. The Worker then processes the Message. It will be a constant flow of Messages. As long as Files came into the Source Directory the Files will be processed.
To implement different kind of Workers i added a common Interface for those workers.
My Question is now would it be better to put the Message Class, Worker Interface and Interfaces of the Observer Pattern (Subject and Observer) into the Balancer Project and let the InputSource, Worker directly depend on that Balancer Library. (First Image)
Or would it be a better approach to introduce something like a Core Library where i place those Class and Interfaces? In that case the Balancer, InputSource and Worker are depend on that Core Library. (second Image).
Hope this clearify some of your points.
Another Book i can recommend is the Professional C++ from Marc Gregoire. I think this is also a great book for beginners.
