Symlists extractor

README.md

@@ -27,3 +27,4 @@ Tools for sysadmins.
 * [sympatoldap](utils/sympatoldap/README.md): creates LDAP entries for every list (and its aliases) whose status is open on the LDAP server
 * [splitting_daemons_logs](utils/splitting_daemons_logs/README.md): split Sympa processus logs into separated files
 * [sqlite2mysql](utils/sqlite2mysql/README.md): attempt to convert sqlite to mysql
+* [symlists_extractor](utils/symlists_extractor/README.md) : extract mailing lists how are subscribers to other mailing lists 

utils/symlists_extractor/README.md

@@ -0,0 +1,65 @@
+# CLI utility: symlists_extractor
+
+## Objective
+
+Retrieve from a SYMPA server the list of mailing lists registered as a member of another mailing list and not *included*
+as good practice dictates... ;)
+
+This script uses the [sympasoap](https://gitlab.com/animath/si/py-sympa-soap) , a python sympa SOAP wrapper library to work.
+
+## Use 
+* Have python installed  (version >=v3.10)
+* Create a virtual environment: 
+  * ``$ python -m venv .venv``
+* To use it: 
+  * ``$ source .venv/bin/activate`` (Linux)
+  * ``$ call C:\path\to\project\env_project_name\Scripts\activate.bat`` (windows)
+  * or ``C:\path\to\project\env_project_name\Scripts\Activate.ps1`` (windows powershell)
+* Install the dependencies of the requirements.txt file with pip
+  * ``$ pip install -r requirements.txt``
+* In a terminal, at the root of the script, user the following command :
+
+``
+$ python symlist_extractor.py *arg1 *arg2 *arg3
+``
+
+* The arguments are **MANDATORY** in the following strict order: 
+  * arg1 = server url without **"/wsdl"** at the end (ex: http://nom_de_domaine.fr/sympa)
+  * arg2 = administrator login (ex: listmaster@sympa.nom_de_domaine.fr)
+  * arg3 = administrator password (ex: mon_beau_mdp)
+
+**/!\ Note 1:**
+* More than 3 arguments => not taken into account 
+* Less than 3 arguments => the script will be in error
+
+After a few minutes, you will get a JSON file with the following structure in the script folder: 
+
+**/!\ Note 2:**
+The dictionary keys are the existing mailing lists on your SYMPA server and the values are the mailing lists that
+are subscribed to the list and are not included in a data source.
+
+```json=
+{
+     "security@domain_name.fr": [ 
+          "abuse@domain_name.fr",
+          "report@domain_name.fr",
+          "....",
+     ]  
+}
+```
+
+## Changlog
+
+v0.0.1: initial release
+
+## Licence
+
+GPLv3
+
+## Author
+
+Nicolas.renard_at_uha.fr
+
+## Original source
+
+* [github source project](https://github.com/nicorenard/symlists_extractor)

utils/symlists_extractor/symlists_extractor.py

@@ -0,0 +1,99 @@
+import json
+import os
+import sys
+from sympasoap import Client
+from sympasoap.client import Fault
+
+
+class SympaController:
+
+    def __init__(self, url: str, login: str, password: str):
+        self.sympa_url = url
+        self.login = login
+        self.password = password
+        self.client = SympaController.get_connect_to_sympa(self)
+
+    def get_connect_to_sympa(self) -> Client:
+        """
+        Méthode de connexion au serveur Sympa.
+
+        :returns: Un client de connexion
+        """
+        try:
+            client = Client(self.sympa_url)
+            client.login(self.login, self.password)
+            return client
+        except Fault as e:
+            msg = f"Erreur, pas de connexion à Sympa : {str(e)}."
+            raise Fault(msg)
+
+    def get_list_in_list(self) -> None:
+        """
+        Méthode pour récupérer des listes de diffusion comme abonnées dans des listes de diffusion.
+        """
+        conn = self.client
+        sympa_lists = []
+        # get all lists
+        all_lists = conn.all_lists()
+        if not len(all_lists) > 0:
+            print(f"Warning - This is probably not an error but the result is empty.")
+            sys.exit(1)
+        all_lists_clean = [mailing_list.list_address for mailing_list in all_lists]
+
+        # get all subscribers
+        for mailing_list in all_lists_clean:
+            subscribers = conn.get_subscribers(mailing_list, emails_only=True)
+            datas = {
+                "name": mailing_list,
+                "members": subscribers,
+            }
+            sympa_lists.append(datas)
+        list_in_list = []
+
+        # cleaning subscribers
+        for mailing_list in all_lists_clean:
+            for list_element in sympa_lists:
+                members = list_element.get("members")
+                if mailing_list in members:
+                    datas = {
+                        "list_address": list_element.get("name"),
+                        "list_hidden_in_list": mailing_list
+                    }
+                    list_in_list.append(datas)
+
+        # merging infos
+        combined_dict = {}
+        for item in list_in_list:
+            list_address = item["list_address"]
+            list_hidden_in_list = item["list_hidden_in_list"]
+            if list_address not in combined_dict:
+                combined_dict[list_address] = []
+            combined_dict[list_address].append(list_hidden_in_list)
+
+        # report
+        file_name = "lists_hidden_in_lists.json"
+        file_path = os.path.join(file_name)
+        folder = os.path.join(os.path.abspath(os.path.dirname(__file__)), file_path)
+        try:
+            with open(folder, "w") as file:
+                json.dump(combined_dict, file, indent=2)
+        except FileExistsError as e:
+            msg = f"Error, can't access to the JSON file: {str(e)}"
+            raise FileExistsError(msg)
+
+
+def main(kwargs: list[str]) -> None:
+    # sympa env
+    sympa_url = kwargs[0]
+    sympa_login = kwargs[1]
+    sympa_password = kwargs[2]
+    sc = SympaController(sympa_url, sympa_login, sympa_password)
+    print("Research in run....")
+    sc.get_list_in_list()
+    print("Research done.\nA JSON file was generated and is available in the script folder.")
+    sys.exit(0)
+
+
+if __name__ == "__main__":
+    args = sys.argv[1:4]
+    main(args)