initial (44cbbee5) | Commits | Mateusz Kopeć / DistSys

Browse Code »

Commit 44cbbee54dc5c5b714b914d91509f922cd9e7bed

Authored by Mateusz Kopeć 12 years ago

0 parents

master

initial

Inline Side-by-side

Showing 68 changed files with 7693 additions and 0 deletions

README 0 → 100644

View file @44cbbee

	1	+++ a/README
	1	+Instrukcja instalacji systemu zarządzania plikami.
	2	+
	3	+
	4	+W poniższej instrukcji zakładamy, że czytelnik posiada podstawową
	5	+wiedzę na temat administracji serwera svn. Zakładamy, że
	6	+po stronie serwera zainstalowane są następujące narzędzia:
	7	+
	8	+* subversion
	9	+* python (2.6 lub 2.7)
	10	+* pysvn
	11	+
	12	+GUI ułatwiające pobieranie i wysyłanie zaanotowanych plików
	13	+zbudowane jest na bazie:
	14	+
	15	+* python
	16	+* wxPython
	17	+* wxGlade
	18	+
	19	+Przygotowana została binarna wersja GUI pod system Windows,
	20	+dzięki czemu użytkownicy systemu nie muszą posiadać
	21	+zainstalowanych powyższych narzedzi.
	22	+
	23	+
	24	+------------------------------------------------------------------
	25	+
	26	+
	27	+Razem z instrukcją README administrator systemu powinien otrzymać:
	28	+
	29	+* Moduł dfs -- implementację systemu zarządzania plikami,
	30	+* Katalog scripts -- zestaw skryptów ułatwiających tworzenie i
	31	+ zarządzanie systemem,
	32	+* Moduł gui -- implementację GUI dla użytkowników systemu.
	33	+* Katalog z przykładową konfiguracją serwera
	34	+ (pliki $SERVER.cfg oraz $USERS.cfg),
	35	+* Katalog z przykładową konfiguracją klienta GUI (plik $GUI.cfg).
	36	+
	37	+Ponadto, administrator powinien wygenerować klucze szyfrujące
	38	+połączenie klient-serwer -- pliki key.pem i cert.pem.
	39	+Odpowiednie informacje na temat generowania kluczy można znaleźć
	40	+pod adresem:
	41	+
	42	+http://docs.python.org/dev/library/ssl.html#certificates
	43	+
	44	+
	45	+==================================================================
	46	+
	47	+
	48	+1) Zakładanie nowego repozytorium.
	49	+
	50	+
	51	+Na serwerze, w katalogu, w którym ma znajdować się repozytorium,
	52	+należy wykonać komendę:
	53	+
	54	+$ svnadmin create $REPO.PATH
	55	+
	56	+W katalogu konfiguracyjnym repozytorium ($REPO.PATH/conf/) należy
	57	+zmodyfikować dwa pliki:
	58	+
	59	+i) svnserve.conf
	60	+
	61	+...
	62	+### and "none". The sample settings below are the defaults.
	63	+anon-access = none
	64	+auth-access = write
	65	+### The password-db option controls the location of the password
	66	+...
	67	+### Uncomment the line below to use the default password file.
	68	+password-db = passwd
	69	+### The authz-db option controls the location of the authorization
	70	+...
	71	+
	72	+ii) passwd
	73	+
	74	+W pliku passwd należy wpisać login i hasło, które wykorzystywane
	75	+będą po stronie serwera do zatwierdzania (commit-owania) zmian
	76	+w repozytorium, np.:
	77	+
	78	+...
	79	+machine = 95mnf83m-SDT%@vf
	80	+...
	81	+
	82	+Ten sam login i hasło należy podać w konfiguracji serwera systemu
	83	+zarządzania plikami, o czym w punkcie 2). W pliku passwd można
	84	+podać również hasła dla innych użytkowników repozytorium -- więcej
	85	+na ten temat można znaleźć w dokumentacji subversion.
	86	+
	87	+Hasła użytkowników systemu (anotatorów) również uzupełniane są w
	88	+konfiguracji systemu zarządzania plikami opisanej w 2).
	89	+
	90	+
	91	+------------------------------------------------------------------
	92	+
	93	+
	94	+Na serwerze, w innym katalogu niż zostało założone repozytorium,
	95	+należy utworzyć kopię roboczą repozytorium, z której korzystać
	96	+będzie system zarządzania plikami. W tym celu należy wpisać:
	97	+
	98	+$ svn co file://$REPO.PATH $COPY.PATH
	99	+Pobrano wersję 0.
	100	+
	101	+Gdzie $REPO.PATH to ścieżka repozytorium, a $COPY.PATH to
	102	+docelowy katalog kopii roboczej.
	103	+
	104	+
	105	+UWAGA: Zamiast ścieżki postaci file://$REPO.PATH, można również
	106	+podać adres w postaci svn://$REPO.ADDR -- adres, który jest
	107	+dostępny dla użytkowników repozytorium pracujących poza
	108	+serwerem. Ponadto, kopię roboczą można założyć na innym
	109	+serwerze niż repozytorium. Kopia robocza musi jednak znajdować
	110	+się na tej samej maszynie, na której będzie później uruchamiany
	111	+serwer systemu zarządzania plikami. Wadą tego rozwiązania
	112	+może być wolniejsze działanie systemu.
	113	+
	114	+
	115	+Pustą kopię roboczą należy na wstępie wypełnić strukturą katalogów
	116	+dla systemu zarządzania plikami. Służy do tego skrypt
	117	+fill_empty_repo.sh znajdujący się w katalogu scripts dostarczonym
	118	+razem z tą instrukcją. Skrypt wywołuje się w następujący sposób:
	119	+
	120	+$ ./fill_empty_repo.sh $COPY.PATH
	121	+
	122	+Struktura katalogów powinna wyglądać następująco:
	123	+
	124	+$COPY.PATH/
	125	+ new <- katalog z nowymi plikami
	126	+ annotation/ <- wyniki anotacji plików
	127	+ A
	128	+ B
	129	+ adjudication <- wyniki super-anotacji plików
	130	+ db.xml <- baza danych z bierzącymi informacjami
	131	+ dotyczącymi anotacji
	132	+
	133	+Na koniec należy watwierdzić zmiany wykonane w kopii roboczej:
	134	+
	135	+$ svn add $COPY.PATH/*
	136	+$ svn commit --username=machine --password=95mnf83m-SDT%@vf -m "struktura"
	137	+
	138	+
	139	+UWAGA: W katalogach annotation/A, annotation/B oraz adjudication
	140	+znajdują się wysłane przez anotatorów na serwer pliki. Nie oznacza
	141	+to jednak, że anotacja tych plików przez ich posiadaczy zakończyła
	142	+się. Taką informację można uzyskać na podstawie bazy danych
	143	+(pliki, których anotacja/super-anotacja zakończyła się, będą
	144	+uzupełnione o element <checkinDate> w bazie danych).
	145	+
	146	+
	147	+==================================================================
	148	+
	149	+
	150	+2) Konfiguracja systemu zarządzania plikami.
	151	+
	152	+
	153	+Konfiguracja systemu odbywa się w trzech plikach:
	154	+* $SERVER.cfg
	155	+* $USERS.cfg
	156	+* $PASSWD
	157	+
	158	+Razem z instrukcją i implementacją serwera powinny zostać
	159	+dostarczone również przykładowe pliki konfiguracyjne, w których
	160	+można znaleźć dokładniejsze informacje na temat konfiguracji
	161	+serwera.
	162	+
	163	+i) $SERVER.cfg
	164	+
	165	+Plik $SERVER.cfg jest głównym plikiem konfiguracyjnym i jedynym
	166	+wczytywanym bezpośrednio przez serwer systemu (znajdują się
	167	+w nim również ścieżki do pozostałych dwóch plików konfiguracyjnych
	168	+-- $USERS.cfg i $PASSWD).
	169	+
	170	+ii) $USERS.cfg
	171	+
	172	+Plik zawiera informacje o uprawnieniach do super-anotacji oraz
	173	+o limitach pobierania plików.
	174	+
	175	+ii) $PASSWD
	176	+
	177	+W pliku znajdują się hasła określone dla użytkowników systemu
	178	+(anotatorów). Poszczególne linie tego pliku mają postać:
	179	+login = hasło
	180	+
	181	+
	182	+==================================================================
	183	+
	184	+
	185	+3) Używanie serwera systemu zarzadznania plikami.
	186	+
	187	+
	188	+Do uruchamiania serwera służy narzędzie server.py w module dfs.
	189	+Serwer uruchamia się następującą komendą:
	190	+
	191	+$ python server.py $SERVER.cfg
	192	+
	193	+Ewentualne błędy będą wypisywane na standardowe wyjście.
	194	+
	195	+
	196	+------------------------------------------------------------------
	197	+
	198	+
	199	+Karmienie systemu nowymi plikami przeznaczonymi do anotacji.
	200	+
	201	+Do tego celu służy skrypt add_files.py w katalogu scripts.
	202	+
	203	+$ python add_files.py $SERVER.cfg PATH1 PATH2 ...
	204	+
	205	+gdzie PATH1, PATH2, ... to ścieżki do plików, które mają zostać
	206	+dodane do repozytorium. Zmiany nie są automatycznie commit-owane,
	207	+więc trzeba użyć komendy svn commit do ich zatwierdzenia:
	208	+
	209	+UWAGA: Dodawanie plików, jak również jakiekolwiek prace
	210	+administracyje w repozytorium systemu powinny być przeprowadzane
	211	+przy wyłączonym serwerze server.py. Serwer można w sposób
	212	+bezpieczny wyłączyć wysyłając sygnał SIGINT (np. Control-C)
	213	+(Sygnał zostanie zignorowany, jeśli odbywa się komunikacja
	214	+z serwerem).
	215	+
	216	+
	217	+==================================================================
	218	+
	219	+
	220	+4) Graficzny interfejs systemu zarządzania plikami.
	221	+
	222	+
	223	+Implementacja GUI znajduje się w module gui dostarczonym
	224	+razem z instrukcją. GUI przeznaczone jest dla użytkowników
	225	+systemu, czyli anotatorów.
	226	+
	227	+
	228	+------------------------------------------------------------------
	229	+
	230	+
	231	+Konfiguracja GUI znajduje się w katalogu gui/data w pliku
	232	+global.cfg. W tym samym katalogu powinien znajdować się
	233	+wygenerowany przez administratora klucz publiczny (cert.pem).
	234	+Dokładniejsze informacje na temat konfiguracji znajdują się w
	235	+przykładowym pliku konfiguracyjnym gui/data/global.cfg.
	236	+
	237	+
	238	+------------------------------------------------------------------
	239	+
	240	+
	241	+GUI uruchamia się komendą:
	242	+
	243	+$ python manager.py
...	...

dfs/__init__.py 0 → 100644

View file @44cbbee

	1	+++ a/dfs/__init__.py
...	...

dfs/config.py 0 → 100644

View file @44cbbee

	1	+++ a/dfs/config.py
	1	+# -- coding: utf-8 --
	2	+import re
	3	+import os
	4	+
	5	+SECTION = re.compile("^\[(.*)\]$")
	6	+SUBSECTION = re.compile("^\{(.*)\}$")
	7	+ENTRY = re.compile("^([^=\s]+)\s=\s([^=]+)\s*$")
	8	+
	9	+def cut_comment(s):
	10	+ return s.split("#")[0]
	11	+
	12	+class Config:
	13	+
	14	+ def __init__(self, path):
	15	+ section = None
	16	+ subsection = None
	17	+ self.d = {}
	18	+ with open(path) as f:
	19	+ for line in f:
	20	+ line = cut_comment(line).strip()
	21	+
	22	+ m = SECTION.search(line)
	23	+ if m is not None:
	24	+ section = m.group(1)
	25	+ subsection = None
	26	+ continue
	27	+
	28	+ m = SUBSECTION.search(line)
	29	+ if m is not None:
	30	+ subsection = m.group(1)
	31	+ continue
	32	+
	33	+ m = ENTRY.search(line)
	34	+ if m is not None:
	35	+ key = m.group(1)
	36	+ val = m.group(2)
	37	+ if subsection == "paths":
	38	+ # print path, val, os.path.join(path, val)
	39	+ val = self.relative_path(val, path)
	40	+ if section is not None:
	41	+ self.d[section + "." + key] = val
	42	+ else:
	43	+ self.d[key] = val
	44	+
	45	+ def relative_path(self, path, ref):
	46	+ return os.path.join(os.path.dirname(ref), path)
	47	+
	48	+ def __getitem__(self, key):
	49	+ return self.d[key]
...	...

dfs/database.py 0 → 100644

View file @44cbbee

	1	+++ a/dfs/database.py
	1	+# -- coding: utf-8 --
	2	+import time
	3	+from datetime import date, datetime
	4	+from collections import defaultdict
	5	+
	6	+from xml.etree.ElementTree import Element, ElementTree, tostring
	7	+import xml.etree.ElementTree as ET
	8	+
	9	+class Database:
	10	+
	11	+ def __init__(self, db, anno_per_file, from_file=True):
	12	+ if from_file:
	13	+ self.db_name = db
	14	+ self.tree = ET.parse(db)
	15	+ self.root = self.tree.getroot()
	16	+ else:
	17	+ self.db_name = None
	18	+ self.tree = None
	19	+ self.root = ET.fromstring(db)
	20	+
	21	+ self.anno_per_file = anno_per_file
	22	+ self.make_index()
	23	+
	24	+ def owned(self, ann_elem):
	25	+ return not self.finished(ann_elem) and not self.returned(ann_elem)
	26	+
	27	+ def finished(self, ann_elem):
	28	+ return ann_elem.find("checkinDate") is not None
	29	+
	30	+ def returned(self, ann_elem):
	31	+ return ann_elem.find("return") is not None
	32	+
	33	+ def get_reason(self, ann_elem):
	34	+ reas = ann_elem.find("return").find("reason").text
	35	+ if reas is None:
	36	+ reas = ""
	37	+ return reas
	38	+
	39	+ def rejected(self, file_elem):
	40	+ return file_elem.find("rejected") is not None
	41	+
	42	+ def fixed(self, ann_elem):
	43	+ ret = ann_elem.find("return")
	44	+ if ret is not None and ret.find("fixed") is not None:
	45	+ return True
	46	+ return False
	47	+
	48	+ def ann_elem_for_user(self, file_elem, user):
	49	+ idx = 0
	50	+ for ann_elem in file_elem.findall("ann"):
	51	+ if ann_elem.find("annName").text == user:
	52	+ return ann_elem, idx
	53	+ idx = idx + 1
	54	+ return None, None
	55	+
	56	+ def sann_elem_for_user(self, file_elem, user):
	57	+ for ann_elem in file_elem.findall("s_ann"):
	58	+ if ann_elem.find("annName").text == user:
	59	+ return ann_elem
	60	+ return None
	61	+
	62	+ def fixed_for_user(self, file_elem, user, is_adj):
	63	+ if is_adj:
	64	+ ann = self.sann_elem_for_user(file_elem, user)
	65	+ else:
	66	+ ann, idx = self.ann_elem_for_user(file_elem, user)
	67	+ if ann is not None:
	68	+ return self.fixed(ann)
	69	+ return False
	70	+
	71	+ def make_index(self):
	72	+ self.file_index = {}
	73	+ self.ann_owned_index = defaultdict(list)
	74	+ self.adj_owned_index = defaultdict(list)
	75	+
	76	+ for file_elem in self.root:
	77	+ name_elem = file_elem.find("name")
	78	+ if name_elem == None:
	79	+ raise Exception("No name assigned to a file element !")
	80	+ self.file_index[name_elem.text] = file_elem
	81	+
	82	+ if self.rejected(file_elem):
	83	+ continue
	84	+
	85	+ for ann_elem in file_elem.findall("ann"):
	86	+ if self.owned(ann_elem):
	87	+ ann_name = ann_elem.find("annName").text
	88	+ self.ann_owned_index[ann_name].append(file_elem)
	89	+
	90	+ for ann_elem in file_elem.findall("s_ann"):
	91	+ if self.owned(ann_elem):
	92	+ ann_name = ann_elem.find("annName").text
	93	+ self.adj_owned_index[ann_name].append(file_elem)
	94	+
	95	+ def fix(self, ann_elem):
	96	+ if not self.returned(ann_elem):
	97	+ raise Exception("File not returned in database!")
	98	+ if self.fixed(ann_elem):
	99	+ raise Exception("File already fixed in database!")
	100	+
	101	+ ret = ann_elem.find("return")
	102	+ fixed = ET.SubElement(ret, "fixed")
	103	+
	104	+ def add(self, file_name):
	105	+ if self.file_index.has_key(file_name):
	106	+ raise Exception("File name already in database !")
	107	+ file_elem = ET.SubElement(self.root, "file")
	108	+ name_elem = ET.SubElement(file_elem, "name")
	109	+ name_elem.text = file_name
	110	+ date_elem = ET.SubElement(file_elem, "addDate")
	111	+ date_elem.text = time.strftime("%c")
	112	+ self.file_index[file_name] = file_elem
	113	+
	114	+ def remove(self, file_name):
	115	+ if not self.file_index.has_key(file_name):
	116	+ raise Exception("File name not in database !")
	117	+ file_elem = self.file_index[file_name]
	118	+ self.root.remove(file_elem)
	119	+ self.make_index()
	120	+
	121	+ def reject(self, file_name, reason):
	122	+ if not self.file_index.has_key(file_name):
	123	+ raise Exception("File name not in database !")
	124	+ file_elem = self.file_index[file_name]
	125	+ if self.rejected(file_elem):
	126	+ raise Exception("File already rejected !")
	127	+ rej_elem = ET.SubElement(file_elem, "rejected")
	128	+ rej_elem.text = reason
	129	+ self.make_index()
	130	+
	131	+ # For prettyprint, used in 'save' method.
	132	+ def indent(self, elem, level=0):
	133	+ i = "\n" + level * " "
	134	+ if len(elem):
	135	+ if not elem.text or not elem.text.strip():
	136	+ elem.text = i + " "
	137	+ if not elem.tail or not elem.tail.strip():
	138	+ elem.tail = i
	139	+ for elem in elem:
	140	+ self.indent(elem, level + 1)
	141	+ if not elem.tail or not elem.tail.strip():
	142	+ elem.tail = i
	143	+ else:
	144	+ if level and (not elem.tail or not elem.tail.strip()):
	145	+ elem.tail = i
	146	+
	147	+ def save(self):
	148	+ if not self.db_name:
	149	+ raise Exception("XML not read from file !")
	150	+
	151	+ self.indent(self.root)
	152	+ self.tree.write(self.db_name, encoding="utf-8")
	153	+
	154	+ def upload_prevention(self, file_name, user, is_super):
	155	+ if not self.file_index.has_key(file_name):
	156	+ return "Filename " + file_name.encode('utf-8') + " not known by server."
	157	+ file_elem = self.file_index[file_name]
	158	+ if self.rejected(file_elem):
	159	+ return "File rejected: " + file_name.encode('utf-8')
	160	+
	161	+ if is_super:
	162	+ ann_elem = file_elem.find("s_ann")
	163	+ if ann_elem is None:
	164	+ return "No superannotator assigned to file " + file_name.encode('utf-8')
	165	+
	166	+ if user != ann_elem.find("annName").text:
	167	+ return "Different superannotator assigned to file " + file_name.encode('utf-8')
	168	+
	169	+ if ann_elem.find("checkinDate") is not None:
	170	+ return "File " + file_name.encode('utf-8') + " already checked in!"
	171	+
	172	+ return None
	173	+
	174	+ else:
	175	+ annotations = file_elem.findall("ann")
	176	+ owners = map(lambda ann: ann.find("annName").text, annotations)
	177	+ if not user in owners:
	178	+ return "User " + user + " doesn't own the file " + file_name.encode('utf-8')
	179	+
	180	+ i = owners.index(user)
	181	+ ann_elem = annotations[i]
	182	+ if ann_elem.find("checkinDate") is not None:
	183	+ return "User " + user + " already checked in the file " + file_name.encode('utf-8')
	184	+
	185	+ return None
	186	+
	187	+ def upload_dest(self, file_name, user):
	188	+ """Upload destination (annName element and ID -- 1 or 2) directory."""
	189	+ file_elem = self.file_index[file_name]
	190	+ annotations = file_elem.findall("ann")
	191	+
	192	+ owners = map(lambda ann: ann.find("annName").text, annotations)
	193	+ exc = Exception("Cannot upload " + file_name.encode('utf-8')
	194	+ + " file by " + user + " annotator !")
	195	+ if not user in owners:
	196	+ raise exc
	197	+
	198	+ i = owners.index(user)
	199	+ ann_elem = annotations[i]
	200	+ if ann_elem.find("checkinDate") is not None:
	201	+ raise exc
	202	+
	203	+ return (ann_elem, i)
	204	+
	205	+ def upload_id(self, file_name, user):
	206	+ return self.upload_dest(file_name, user)[1]
	207	+
	208	+ def return_file(self, file_name, user, reason):
	209	+ (ann_elem, i) = self.upload_dest(file_name, user)
	210	+ ret_elem = ET.SubElement(ann_elem, "return")
	211	+ date_elem = ET.SubElement(ret_elem, "date")
	212	+ date_elem.text = time.strftime("%c")
	213	+ date_elem = ET.SubElement(ret_elem, "reason")
	214	+ if reason is None:
	215	+ reason = ""
	216	+ date_elem.text = reason
	217	+ return i
	218	+
	219	+ def return_file_prim(self, file_name, user, reason):
	220	+ file_elem = self.file_index[file_name]
	221	+ ann_elem = file_elem.find("s_ann")
	222	+
	223	+ if (user != ann_elem.find("annName").text or
	224	+ ann_elem.find("checkinDate") is not None):
	225	+ raise Exception("Cannot upload " + file_name.encode('utf-8')
	226	+ + " file by " + user + " annotator !")
	227	+
	228	+ ret_elem = ET.SubElement(ann_elem, "return")
	229	+ date_elem = ET.SubElement(ret_elem, "date")
	230	+ date_elem.text = time.strftime("%c")
	231	+ date_elem = ET.SubElement(ret_elem, "reason")
	232	+ if reason is None:
	233	+ reason = ""
	234	+ date_elem.text = reason
	235	+
	236	+ def upload(self, file_name, user):
	237	+ (ann_elem, i) = self.upload_dest(file_name, user)
	238	+ date_elem = ET.SubElement(ann_elem, "checkinDate")
	239	+ date_elem.text = time.strftime("%c")
	240	+ return i
	241	+
	242	+ def upload_prim(self, file_name, user):
	243	+ file_elem = self.file_index[file_name]
	244	+ ann_elem = file_elem.find("s_ann")
	245	+
	246	+ if (user != ann_elem.find("annName").text or self.finished(ann_elem)):
	247	+ raise Exception("Cannot upload " + file_name.encode('utf-8')
	248	+ + " file by " + user + " annotator !")
	249	+
	250	+ date_elem = ET.SubElement(ann_elem, "checkinDate")
	251	+ date_elem.text = time.strftime("%c")
	252	+
	253	+ def download(self, file_name, user):
	254	+ file_elem = self.file_index[file_name]
	255	+
	256	+ annotations = file_elem.findall("ann")
	257	+ owners = map(lambda ann: ann.find("annName").text, annotations)
	258	+
	259	+ if self.fixed_for_user(file_elem, user, False):
	260	+ ann, idx = self.ann_elem_for_user(file_elem, user)
	261	+ date = ann.find("return").find("date").text
	262	+ ann.remove(ann.find("return"))
	263	+
	264	+ date_elem = ET.SubElement(ann, "returnDate")
	265	+ date_elem.text = date
	266	+
	267	+ date_elem = ET.SubElement(ann, "checkoutDate")
	268	+ date_elem.text = time.strftime("%c")
	269	+
	270	+ return idx
	271	+
	272	+ else:
	273	+ if user in owners or len(owners) + 1 > self.anno_per_file:
	274	+ raise Exception("Cannot set " + user + " annotator to '"
	275	+ + file_name.encode('utf-8') + "' file !")
	276	+
	277	+ ann_elem = ET.SubElement(file_elem, "ann")
	278	+ ann_name_elem = ET.SubElement(ann_elem, "annName")
	279	+ ann_name_elem.text = user
	280	+
	281	+ date_elem = ET.SubElement(ann_elem, "checkoutDate")
	282	+ date_elem.text = time.strftime("%c")
	283	+
	284	+ return None
	285	+
	286	+ def download_prim(self, file_name, user):
	287	+ file_elem = self.file_index[file_name]
	288	+
	289	+ if self.fixed_for_user(file_elem, user, True):
	290	+ sann = self.sann_elem_for_user(file_elem, user)
	291	+ date = sann.find("return").find("date").text
	292	+ sann.remove(sann.find("return"))
	293	+
	294	+ date_elem = ET.SubElement(sann, "returnDate")
	295	+ date_elem.text = date
	296	+
	297	+ date_elem = ET.SubElement(sann, "checkoutDate")
	298	+ date_elem.text = time.strftime("%c")
	299	+
	300	+ return True
	301	+
	302	+ else:
	303	+ if len(file_elem.findall("s_ann")) > 0:
	304	+ raise Exception("Cannot set " + user + " adjudicator to '"
	305	+ + file_name.encode('utf-8') + "' file !")
	306	+
	307	+ ann_elem = ET.SubElement(file_elem, "s_ann")
	308	+ ann_name_elem = ET.SubElement(ann_elem, "annName")
	309	+ ann_name_elem.text = user
	310	+
	311	+ date_elem = ET.SubElement(ann_elem, "checkoutDate")
	312	+ date_elem.text = time.strftime("%c")
	313	+
	314	+ return False
	315	+
	316	+ def for_annotation(self, user):
	317	+ priority = []
	318	+ result = []
	319	+ for file_elem in self.root:
	320	+ if self.rejected(file_elem) or file_elem.find("s_ann") is not None:
	321	+ continue
	322	+
	323	+ anns = file_elem.findall("ann")
	324	+ owners = map(lambda ann: ann.find("annName").text, anns)
	325	+ if len(owners) < self.anno_per_file and user not in owners:
	326	+ result.append(file_elem.find("name").text)
	327	+
	328	+ if self.fixed_for_user(file_elem, user, False):
	329	+ priority.append(file_elem.find("name").text)
	330	+
	331	+ return priority, result
	332	+
	333	+ def for_adjudication(self, user):
	334	+ priority = []
	335	+ result = []
	336	+ for file_elem in self.root:
	337	+ if self.rejected(file_elem):
	338	+ continue
	339	+
	340	+ anns = file_elem.findall("ann")
	341	+ owners = map(lambda ann: ann.find("annName").text, anns)
	342	+ checked = map(lambda ann: ann.find("checkinDate") != None, anns)
	343	+ if (len(owners) == self.anno_per_file
	344	+ and file_elem.find("s_ann") is None
	345	+ and user not in owners
	346	+ and all(checked)):
	347	+ result.append(file_elem.find("name").text)
	348	+
	349	+ if self.fixed_for_user(file_elem, user, True):
	350	+ priority.append(file_elem.find("name").text)
	351	+
	352	+ return priority, result
	353	+
	354	+ def owns_normal(self, user):
	355	+ return [ file_elem.find("name").text
	356	+ for file_elem
	357	+ in self.ann_owned_index[user] ]
	358	+
	359	+ def owns_super(self, user):
	360	+ return [ file_elem.find("name").text
	361	+ for file_elem
	362	+ in self.adj_owned_index[user] ]
	363	+
	364	+ def owns(self, user):
	365	+ return self.owns_normal(user) + self.owns_super(user)
	366	+
	367	+ def finished_count(self, user):
	368	+ n = 0
	369	+ for file_elem in self.root:
	370	+
	371	+ if self.rejected(file_elem):
	372	+ continue
	373	+
	374	+ items = file_elem.findall("ann")
	375	+ items.extend(file_elem.findall("s_ann"))
	376	+ for ann_elem in items:
	377	+ ann_name = ann_elem.find("annName").text
	378	+ if ann_name == user and self.finished(ann_elem) and not self.returned(ann_elem):
	379	+ n += 1
	380	+
	381	+ return n
...	...

dfs/msg/__init__.py 0 → 100644

View file @44cbbee

	1	+++ a/dfs/msg/__init__.py
...	...

dfs/msg/message.py 0 → 100644

View file @44cbbee

	1	+++ a/dfs/msg/message.py
	1	+# -- coding: utf-8 --
	2	+__all__ = ["decode", "Message", "OkMessage", "KoMessage", "DownloadRequest"
	3	+ , "UploadRequest", "DownloadPrimRequest", "ClientDone", "ServerDone"
	4	+ , "NumMessage", "BoolMessage", "ReturnRequest", "CheckoutRequest",
	5	+ "CheckinRequest", "StatsRequest"]
	6	+
	7	+
	8	+class Message(object):
	9	+
	10	+ """
	11	+ Base class for SSL communication messages.
	12	+ """
	13	+
	14	+ def __init__(self, contents=""):
	15	+ self.contents = contents
	16	+
	17	+ def get_contents(self):
	18	+ return self.contents
	19	+
	20	+ @classmethod
	21	+ def from_contents(cls, contents):
	22	+ return cls(contents)
	23	+
	24	+ def encode(self):
	25	+ """
	26	+ Prepare message to be sent over network.
	27	+ WARNING: Do not override this method.
	28	+ """
	29	+ # Add "-" on the beggining to prevent
	30	+ # message contents from being void.
	31	+ return self.msg_type, "-" + self.get_contents()
	32	+
	33	+
	34	+class NumMessage(Message):
	35	+
	36	+ def __init__(self, n):
	37	+ self.number = n
	38	+
	39	+ def get_contents(self):
	40	+ return str(self.number)
	41	+
	42	+ @classmethod
	43	+ def from_contents(cls, contents):
	44	+ return cls(int(contents))
	45	+
	46	+ def get_number(self):
	47	+ return self.number
	48	+
	49	+
	50	+class BoolMessage(Message):
	51	+
	52	+ def __init__(self, b):
	53	+ self.b = b
	54	+
	55	+ def get_contents(self):
	56	+ return "1" if self.b else "0"
	57	+
	58	+ @classmethod
	59	+ def from_contents(cls, contents):
	60	+ return cls(bool(int(contents)))
	61	+
	62	+ def get_boolean(self):
	63	+ return self.b
	64	+
	65	+class ReturnRequest(NumMessage):
	66	+ pass
	67	+
	68	+class CheckoutRequest(Message):
	69	+ pass
	70	+
	71	+class CheckinRequest(NumMessage):
	72	+ pass
	73	+
	74	+class DownloadRequest(NumMessage):
	75	+ pass
	76	+
	77	+class DownloadPrimRequest(NumMessage):
	78	+ pass
	79	+
	80	+class UploadRequest(NumMessage):
	81	+ pass
	82	+
	83	+class OkMessage(Message):
	84	+ pass
	85	+
	86	+class KoMessage(Message):
	87	+ pass
	88	+
	89	+class StatsRequest(Message):
	90	+ pass
	91	+
	92	+class ServerDone(Message):
	93	+ pass
	94	+
	95	+class ClientDone(Message):
	96	+ pass
	97	+
	98	+
	99	+def subclasses(cls):
	100	+ yield cls
	101	+ for child in cls.__subclasses__():
	102	+ for desc in subclasses(child):
	103	+ yield desc
	104	+
	105	+def decode(msg_type, contents):
	106	+ for cls in subclasses(Message):
	107	+ if cls.msg_type == msg_type:
	108	+ # Discard first, dummy character;
	109	+ # Confront Message.encode method.
	110	+ return cls.from_contents(contents[1:])
	111	+
	112	+for i, cls in enumerate(subclasses(Message)):
	113	+ cls.msg_type = i
...	...

dfs/msg/stream.py 0 → 100644

View file @44cbbee

	1	+++ a/dfs/msg/stream.py
	1	+# -- coding: utf-8 --
	2	+import struct
	3	+
	4	+from message import decode
	5	+
	6	+__all__ = ["read_msg", "write_msg", "EndOfStream"]
	7	+
	8	+class EndOfStream(Exception):
	9	+ pass
	10	+
	11	+class BadMessage(Exception):
	12	+ pass
	13	+
	14	+def read_msg(connstream):
	15	+ encoded = read_encoded(connstream)
	16	+ return decode(*encoded)
	17	+
	18	+def accept_msg(connstream, cls):
	19	+ msg = read_msg(connstream)
	20	+ if type(msg) != cls:
	21	+ raise BadMessage("Expected %s, got %s" %
	22	+ (cls.__name__, type(msg).__name__))
	23	+ return msg
	24	+
	25	+def write_msg(connstream, msg):
	26	+ encoded = msg.encode()
	27	+ write_encoded(connstream, *encoded)
	28	+
	29	+def read_encoded(connstream):
	30	+ _type = read_type(connstream)
	31	+ length = read_length(connstream)
	32	+ content = read_content(connstream, length)
	33	+ return _type, content
	34	+
	35	+def write_encoded(connstream, _type, content):
	36	+ write_type(connstream, _type)
	37	+ write_length(connstream, len(content))
	38	+ write_content(connstream, content)
	39	+
	40	+def read_type(connstream):
	41	+ data = read_all(connstream, 1)
	42	+ return struct.unpack("!B", data)[0]
	43	+
	44	+def write_type(connstream, _type):
	45	+ data = struct.pack("!B", _type)
	46	+ write_all(connstream, data)
	47	+
	48	+def read_length(connstream):
	49	+ data = read_all(connstream, 4)
	50	+ return struct.unpack("!L", data)[0]
	51	+
	52	+def write_length(connstream, n):
	53	+ data = struct.pack("!L", n)
	54	+ write_all(connstream, data)
	55	+
	56	+def read_content(connstream, length):
	57	+ return read_all(connstream, length)
	58	+
	59	+def write_content(connstream, content):
	60	+ write_all(connstream, content)
	61	+
	62	+def write_all(connstream, data):
	63	+ k = connstream.write(data)
	64	+ if k != len(data):
	65	+ raise Exception("Did not send all data !")
	66	+
	67	+def read_all(connstream, k):
	68	+ data = ""
	69	+ while k > 0:
	70	+ # part = connstream.read()
	71	+ part = connstream.read(min(k, 1024))
	72	+ if len(part) == 0:
	73	+ raise EndOfStream
	74	+ k -= len(part)
	75	+ # if k < 0:
	76	+ # raise Exception("Received more than expected !")
	77	+ data += part
	78	+ return data
	79	+
...	...

dfs/repo.py 0 → 100644

View file @44cbbee

	1	+++ a/dfs/repo.py
	1	+# -- coding: utf-8 --
	2	+import os
	3	+import shutil
	4	+import pysvn
	5	+
	6	+class Repo:
	7	+
	8	+ def __init__(self, path, login, passwd):
	9	+ self.path = path
	10	+ def get_login(_realm, _username, _may_save):
	11	+ return True, login, passwd, False
	12	+ self.svn = pysvn.Client()
	13	+ self.svn.callback_get_login = get_login
	14	+ self.svn.update(self.path)
	15	+
	16	+ def add(self, src_path):
	17	+ part_name = os.path.basename(src_path)
	18	+ dest_path = os.path.join(self.new_path(), part_name)
	19	+
	20	+ if (not self.svn.info(dest_path) is None) or (os.path.exists(dest_path)):
	21	+ raise Exception(src_path+" already present in repository.")
	22	+
	23	+ shutil.copy(src_path, dest_path)
	24	+ self.svn.add(dest_path)
	25	+
	26	+ def remove(self, filename, anno_per_file):
	27	+ paths = set()
	28	+ paths.add(os.path.join(self.new_path(), filename))
	29	+ for i in range(anno_per_file):
	30	+ paths.add(self.upload_path(filename, i))
	31	+ paths.add(os.path.join(self.finito_path(), filename))
	32	+
	33	+ removed = []
	34	+ for path in paths:
	35	+ if os.path.exists(path):
	36	+ self.svn.remove(path)
	37	+ removed.append(path)
	38	+
	39	+ return removed
	40	+
	41	+ def db_path(self):
	42	+ return os.path.join(self.path, "db.xml")
	43	+
	44	+ def new_path(self):
	45	+ return os.path.join(self.path, "new")
	46	+
	47	+ def ann_path(self, idx):
	48	+ ids = chr(idx+ord('A'))
	49	+ tail = os.path.join("annotation", ids)
	50	+ return os.path.join(self.path, tail)
	51	+
	52	+ def finito_path(self):
	53	+ return os.path.join(self.path, "adjudication")
	54	+
	55	+ def upload_path(self, file_name, idx):
	56	+ return os.path.join(self.ann_path(idx), file_name)
	57	+
	58	+ def upload_prim_path(self, file_name):
	59	+ return os.path.join(self.finito_path(), file_name)
	60	+
	61	+ def read_data(self, path):
	62	+ with open(path) as src:
	63	+ return src.read()
	64	+
	65	+ def read_contents(self, path, exts):
	66	+ contents = {}
	67	+ for ext in exts:
	68	+ contents[ext] = self.read_data(path + ext)
	69	+ return contents
	70	+
	71	+ def write_data(self, path, data):
	72	+ with open(path, "w") as dest:
	73	+ dest.write(data)
	74	+
	75	+ def write_contents(self, path, contents):
	76	+ for ext, data in contents.iteritems():
	77	+ dest_path = path + ext
	78	+ self.write_data(dest_path, data)
	79	+ if self.svn.info(dest_path) is None:
	80	+ self.svn.add(dest_path)
	81	+
	82	+ def upload(self, file_id, idx, contents):
	83	+ self.write_contents(self.upload_path(file_id, idx), contents)
	84	+
	85	+ def upload_prim(self, file_id, contents):
	86	+ self.write_contents(self.upload_prim_path(file_id), contents)
	87	+
	88	+ def download(self, file_id, exts):
	89	+ return self.read_contents(os.path.join(self.new_path(), file_id), exts)
	90	+
	91	+ def download_prim(self, file_id, exts, anno_per_file):
	92	+ result = []
	93	+ for i in range(anno_per_file):
	94	+ conts = self.read_contents(os.path.join(self.ann_path(i), file_id), exts)
	95	+ result.append(conts)
	96	+ return result
	97	+
	98	+ def checkout(self, file_id, idx, exts):
	99	+ src_paths = [ self.upload_path(file_id, idx)
	100	+ , os.path.join(self.new_path(), file_id) ]
	101	+ for path in src_paths:
	102	+ if all(os.path.exists(path + ext) for ext in exts):
	103	+ return self.read_contents(path, exts)
	104	+
	105	+ def checkout_prim(self, file_id, exts):
	106	+ path = os.path.join(self.finito_path(), file_id)
	107	+ if all(os.path.exists(path + ext) for ext in exts):
	108	+ return self.read_contents(path, exts)
	109	+
	110	+ def commit(self, message):
	111	+ self.svn.checkin(self.path, message, recurse=True)
	112	+
	113	+ def revert(self):
	114	+ self.svn.revert(self.path, recurse=True)
...	...

dfs/server.py 0 → 100644

View file @44cbbee

	1	+++ a/dfs/server.py
	1	+# -- coding: utf-8 --
	2	+import sys
	3	+import os
	4	+import traceback
	5	+import socket, ssl
	6	+import random
	7	+from optparse import OptionParser
	8	+import signal
	9	+
	10	+from utils import validate_user, UserInvalid, send_contents, receive_contents
	11	+from msg.stream import read_msg, write_msg, accept_msg, EndOfStream, BadMessage
	12	+from msg.message import *
	13	+from repo import Repo
	14	+from database import Database
	15	+from config import Config
	16	+
	17	+class SSLServer:
	18	+
	19	+ def __init__(self, host, port, backlog, cert_file, key_file,
	20	+ repository, svn_login, svn_passwd, pass_file,
	21	+ users_file, file_exts, anno_per_file, log=None):
	22	+ """
	23	+ Initialize SSL server.
	24	+
	25	+ params:
	26	+ =======
	27	+ host : str
	28	+ Host name.
	29	+ port : int
	30	+ Port number.
	31	+ backlog : int
	32	+ Maximum number of waiting connections.
	33	+ cert_file : path
	34	+ Public PEM certificate.
	35	+ key_file : path
	36	+ Private key.
	37	+ pass_file : path
	38	+ File with client passwords.
	39	+ users_file : path
	40	+ File with additional users configuration.
	41	+ repository : path
	42	+ Subversion working copy.
	43	+ svn_login : str
	44	+ Subversion login.
	45	+ svn_passwd : str
	46	+ Subversion password.
	47	+ file_exts : [str]
	48	+ List of file extensions.
	49	+ anno_per_file : int
	50	+ Number of normal annotators per file (1 or 2)
	51	+ """
	52	+ self.cert_file = cert_file
	53	+ self.key_file = key_file
	54	+ self.pass_file = pass_file
	55	+ self.users_file = users_file
	56	+ self.wc = Repo(repository, svn_login, svn_passwd)
	57	+ self.file_exts = file_exts
	58	+ if log is not None:
	59	+ self.log = open(log, "a")
	60	+ else:
	61	+ self.log = sys.stdout
	62	+ self.bound = bind_socket(host, port, backlog=backlog)
	63	+ self.serving = None
	64	+ self.anno_per_file = anno_per_file
	65	+
	66	+ def run(self):
	67	+ while True:
	68	+ newsocket, fromaddr = self.bound.accept()
	69	+ self.serving = fromaddr
	70	+ connstream = None
	71	+ try:
	72	+ connstream = ssl.wrap_socket(newsocket,
	73	+ server_side=True,
	74	+ certfile=self.cert_file,
	75	+ keyfile=self.key_file,
	76	+ ssl_version=ssl.PROTOCOL_TLSv1)
	77	+
	78	+ login = validate_user(connstream, self.pass_file)
	79	+ self.serve_client(connstream, login)
	80	+ except UserInvalid as login:
	81	+ print >> self.log, "AUTH ERROR:", login
	82	+ except BadMessage as info:
	83	+ print >> self.log, "BAD MESSAGE ERROR:", info
	84	+ print >> self.log, traceback.format_exc().strip()
	85	+ except EndOfStream:
	86	+ print >> self.log, "UNEXPECTED END OF STREAM:"
	87	+ print >> self.log, traceback.format_exc().strip()
	88	+ except:
	89	+ print >> self.log, "UNEXPECTED ERROR:"
	90	+ print >> self.log, traceback.format_exc().strip()
	91	+ finally:
	92	+ try:
	93	+ if not connstream is None:
	94	+ connstream.shutdown(socket.SHUT_RDWR)
	95	+ except:
	96	+ print >> self.log, "UNEXPECTED ERROR:"
	97	+ print >> self.log, traceback.format_exc().strip()
	98	+ finally:
	99	+ if not connstream is None:
	100	+ connstream.close()
	101	+ self.serving = None
	102	+
	103	+ def exit(self):
	104	+ if self.serving is not None:
	105	+ print ("Client from %s connected to the server"
	106	+ % str(self.serving))
	107	+ else:
	108	+ sys.exit(0)
	109	+
	110	+ def process_msg(self, msg, stream, login):
	111	+ if type(msg) == UploadRequest:
	112	+ self.process_upload_msg(msg, stream, login, checkin=False)
	113	+ elif type(msg) == CheckinRequest:
	114	+ self.process_upload_msg(msg, stream, login, checkin=True)
	115	+ elif type(msg) == CheckoutRequest:
	116	+ self.process_checkout_msg(msg, stream, login)
	117	+ elif type(msg) == DownloadRequest:
	118	+ self.process_download_msg(msg, stream, login)
	119	+ elif type(msg) == DownloadPrimRequest:
	120	+ self.process_download_prim_msg(msg, stream, login)
	121	+ elif type(msg) == ReturnRequest:
	122	+ self.process_return_msg(msg, stream, login)
	123	+ elif type(msg) == StatsRequest:
	124	+ self.process_stats_msg(msg, stream, login)
	125	+ else:
	126	+ print >> self.log, login, "sent:", msg
	127	+
	128	+ def process_return_msg(self, msg, stream, login):
	129	+ reason = accept_msg(stream, Message).get_contents().decode("utf-8")
	130	+
	131	+ usr_cfg = Config(self.users_file)
	132	+ try:
	133	+ user_adj = login in usr_cfg["auth.adjudicators"].split()
	134	+ except KeyError:
	135	+ user_adj = False
	136	+
	137	+ db = Database(self.wc.db_path(), self.anno_per_file)
	138	+ n = msg.get_number()
	139	+ for _ in range(n):
	140	+ file_id = accept_msg(stream, Message).get_contents()
	141	+ is_adj = accept_msg(stream, BoolMessage).get_boolean()
	142	+ if is_adj and not user_adj:
	143	+ write_msg(stream, KoMessage())
	144	+ print >> self.log, "Upload error - user " + login + " tried to upload super annotated file " + file_id + " but is not a super annotator"
	145	+ continue
	146	+ else:
	147	+ error = db.upload_prevention(file_id, login, is_adj)
	148	+ if error is not None:
	149	+ write_msg(stream, KoMessage())
	150	+ print >> self.log, "Upload error by user " + login + ". Details: "+error
	151	+ continue
	152	+
	153	+ write_msg(stream, OkMessage())
	154	+ contents = receive_contents(stream)
	155	+ for key in contents.keys():
	156	+ if key not in self.file_exts:
	157	+ print >> self.log, "Skipped file uploaded by user " + login + ". Filename:"+file_id+key
	158	+ del contents[key]
	159	+
	160	+ if is_adj is False:
	161	+ idx = db.return_file(file_id, login, reason)
	162	+ self.wc.upload(file_id, idx, contents)
	163	+ else:
	164	+ db.return_file_prim(file_id, login, reason)
	165	+ self.wc.upload_prim(file_id, contents)
	166	+
	167	+ db.save()
	168	+ accept_msg(stream, ClientDone)
	169	+ self.wc.commit("return request from %s"
	170	+ % (login))
	171	+ write_msg(stream, ServerDone())
	172	+
	173	+
	174	+ def process_upload_msg(self, msg, stream, login, checkin=False):
	175	+ usr_cfg = Config(self.users_file)
	176	+ try:
	177	+ user_adj = login in usr_cfg["auth.adjudicators"].split()
	178	+ except KeyError:
	179	+ user_adj = False
	180	+
	181	+ db = Database(self.wc.db_path(), self.anno_per_file)
	182	+ n = msg.get_number()
	183	+ for _ in range(n):
	184	+ file_id = accept_msg(stream, Message).get_contents()
	185	+ is_adj = accept_msg(stream, BoolMessage).get_boolean()
	186	+ if is_adj and not user_adj:
	187	+ write_msg(stream, KoMessage())
	188	+ print >> self.log, "Upload error - user " + login + " tried to upload super annotated file " + file_id + " but is not a super annotator"
	189	+ continue
	190	+ else:
	191	+ error = db.upload_prevention(file_id, login, is_adj)
	192	+ if error is not None:
	193	+ write_msg(stream, KoMessage())
	194	+ print >> self.log, "Upload error by user " + login + ". Details: "+error
	195	+ continue
	196	+
	197	+ write_msg(stream, OkMessage())
	198	+ contents = receive_contents(stream)
	199	+ for key in contents.keys():
	200	+ if key not in self.file_exts:
	201	+ print >> self.log, "Skipped file uploaded by user " + login + ". Filename:"+file_id+key
	202	+ del contents[key]
	203	+
	204	+ if is_adj is False:
	205	+ if checkin:
	206	+ idx = db.upload(file_id, login)
	207	+ else:
	208	+ idx = db.upload_id(file_id, login)
	209	+ self.wc.upload(file_id, idx, contents)
	210	+ else:
	211	+ if checkin:
	212	+ db.upload_prim(file_id, login)
	213	+ self.wc.upload_prim(file_id, contents)
	214	+
	215	+ if checkin:
	216	+ db.save()
	217	+ accept_msg(stream, ClientDone)
	218	+ self.wc.commit("%s request from %s"
	219	+ % ("checkin" if checkin else "upload", login))
	220	+ write_msg(stream, ServerDone())
	221	+
	222	+ def process_checkout_msg(self, msg, stream, login):
	223	+ db = Database(self.wc.db_path(), self.anno_per_file)
	224	+
	225	+ ann_files = db.owns_normal(login)
	226	+ write_msg(stream, NumMessage(len(ann_files)))
	227	+ for file_id in ann_files:
	228	+ write_msg(stream, Message(file_id))
	229	+ idx = db.upload_id(file_id, login)
	230	+ contents = self.wc.checkout(file_id, idx, self.file_exts)
	231	+ send_contents(stream, contents)
	232	+
	233	+ adj_files = db.owns_super(login)
	234	+ write_msg(stream, NumMessage(len(adj_files)))
	235	+ write_msg(stream, NumMessage(self.anno_per_file))
	236	+ for file_id in adj_files:
	237	+ write_msg(stream, Message(file_id))
	238	+
	239	+ contents = self.wc.download_prim(file_id, self.file_exts, self.anno_per_file)
	240	+ for conts in contents:
	241	+ send_contents(stream, conts)
	242	+
	243	+ conts3 = self.wc.checkout_prim(file_id, self.file_exts)
	244	+ if conts3 is None:
	245	+ write_msg(stream, BoolMessage(False))
	246	+ else:
	247	+ write_msg(stream, BoolMessage(True))
	248	+ send_contents(stream, conts3)
	249	+
	250	+ accept_msg(stream, ClientDone)
	251	+ #self.wc.commit("checkout request from %s" % login)
	252	+ write_msg(stream, ServerDone())
	253	+
	254	+ def process_stats_msg(self, msg, stream, login):
	255	+ usr_cfg = Config(self.users_file)
	256	+ db = Database(self.wc.db_path(), self.anno_per_file)
	257	+
	258	+ write_msg(stream, NumMessage(db.finished_count(login)))
	259	+
	260	+ accept_msg(stream, ClientDone)
	261	+ write_msg(stream, ServerDone())
	262	+
	263	+ def process_download_msg(self, msg, stream, login):
	264	+ usr_cfg = Config(self.users_file)
	265	+ db = Database(self.wc.db_path(), self.anno_per_file)
	266	+
	267	+ down_num = msg.get_number()
	268	+ owns_num = len(db.owns_normal(login))
	269	+ limit = float('inf')
	270	+ try:
	271	+ limit = int(usr_cfg["limits.%s.annotation" % login])
	272	+ except KeyError:
	273	+ limit = int(usr_cfg["limits.annotation"])
	274	+ if down_num + owns_num > limit:
	275	+ down_num = max(0, limit - owns_num)
	276	+
	277	+ for_ann_fixed, for_ann = db.for_annotation(login)
	278	+ if len(for_ann) + len(for_ann_fixed) == 0:
	279	+ print >> self.log, "User " + login + " has no more files to download for annotation."
	280	+
	281	+ down_files = sample(for_ann_fixed, down_num)
	282	+ down_num = max(0, down_num - len(down_files))
	283	+ down_files = down_files + sample(for_ann, down_num)
	284	+
	285	+ write_msg(stream, NumMessage(len(down_files)))
	286	+
	287	+ for file_id in down_files:
	288	+ fixed_idx = db.download(file_id, login)
	289	+ write_msg(stream, Message(file_id))
	290	+ if fixed_idx is None:
	291	+ contents = self.wc.download(file_id, self.file_exts)
	292	+ else:
	293	+ contents = self.wc.checkout(file_id, fixed_idx, self.file_exts)
	294	+ send_contents(stream, contents)
	295	+ db.save()
	296	+ accept_msg(stream, ClientDone)
	297	+ self.wc.commit("download request from %s" % login)
	298	+ write_msg(stream, ServerDone())
	299	+
	300	+ def process_download_prim_msg(self, msg, stream, login):
	301	+ usr_cfg = Config(self.users_file)
	302	+ try:
	303	+ adjudicators = usr_cfg["auth.adjudicators"].split()
	304	+ except KeyError:
	305	+ adjudicators = []
	306	+ if login in adjudicators:
	307	+ write_msg(stream, OkMessage())
	308	+ else:
	309	+ write_msg(stream, KoMessage())
	310	+ return
	311	+ db = Database(self.wc.db_path(), self.anno_per_file)
	312	+
	313	+ down_num = msg.get_number()
	314	+ owns_num = len(db.owns_super(login))
	315	+ limit = float('inf')
	316	+ try:
	317	+ limit = int(usr_cfg["limits.%s.adjudication" % login])
	318	+ except KeyError:
	319	+ limit = int(usr_cfg["limits.adjudication"])
	320	+ if down_num + owns_num > limit:
	321	+ down_num = max(0, limit - owns_num)
	322	+
	323	+ for_adj_fixed, for_adj = db.for_adjudication(login)
	324	+ if len(for_adj) + len(for_adj_fixed) == 0:
	325	+ print >> self.log, "User " + login + " has no more files to download for superannotation."
	326	+
	327	+ down_files = sample(for_adj_fixed, down_num)
	328	+ down_num = max(0, down_num - len(down_files))
	329	+ down_files = down_files + sample(for_adj, down_num)
	330	+
	331	+ write_msg(stream, NumMessage(len(down_files)))
	332	+ write_msg(stream, NumMessage(self.anno_per_file))
	333	+ for file_id in down_files:
	334	+ write_msg(stream, Message(file_id))
	335	+ fixed = db.download_prim(file_id, login)
	336	+
	337	+ contents = self.wc.download_prim(file_id, self.file_exts, self.anno_per_file)
	338	+ for conts in contents:
	339	+ send_contents(stream, conts)
	340	+
	341	+ if fixed:
	342	+ write_msg(stream, BoolMessage(True))
	343	+ contents = self.wc.checkout_prim(file_id, self.file_exts)
	344	+ send_contents(stream, contents)
	345	+ else:
	346	+ write_msg(stream, BoolMessage(False))
	347	+
	348	+ db.save()
	349	+ accept_msg(stream, ClientDone)
	350	+ self.wc.commit("download' request from %s" % login)
	351	+ write_msg(stream, ServerDone())
	352	+
	353	+ def serve_client(self, connstream, login):
	354	+ msg = read_msg(connstream)
	355	+ try:
	356	+ self.process_msg(msg, connstream, login)
	357	+ except:
	358	+ self.wc.revert()
	359	+ raise
	360	+
	361	+def bind_socket(host, port, backlog=5):
	362	+ bound = socket.socket()
	363	+ bound.bind((host, port))
	364	+ bound.listen(backlog)
	365	+ return bound
	366	+
	367	+def sample(population, n):
	368	+ if len(population) > n:
	369	+ return random.sample(population, n)
	370	+ else:
	371	+ return population
	372	+
	373	+if __name__ == "__main__":
	374	+ optparser = OptionParser(usage="""usage: %prog CONFIG""")
	375	+ (options, args) = optparser.parse_args()
	376	+ if len(args) != 1:
	377	+ optparser.print_help()
	378	+ sys.exit(0)
	379	+
	380	+ cfg = Config(args[0])
	381	+ server = SSLServer(
	382	+ host=cfg["connection.host"],
	383	+ port=int(cfg["connection.port"]),
	384	+ backlog=int(cfg["connection.backlog"]),
	385	+ cert_file=cfg["connection.certfile"],
	386	+ key_file=cfg["connection.keyfile"],
	387	+ repository=cfg["svn.repository"],
	388	+ svn_login=cfg["svn.login"],
	389	+ svn_passwd=cfg["svn.passwd"],
	390	+ pass_file=cfg["users.passfile"],
	391	+ users_file=cfg["users.config"],
	392	+ file_exts=cfg["file_extensions"].split(),
	393	+ anno_per_file=int(cfg["anno_per_file"])
	394	+ )
	395	+
	396	+ def handler(signum, frame):
	397	+ server.exit()
	398	+
	399	+ signal.signal(signal.SIGINT, handler)
	400	+
	401	+ server.run()
...	...

dfs/utils.py 0 → 100644

View file @44cbbee

	1	+++ a/dfs/utils.py
	1	+# -- coding: utf-8 --
	2	+import hashlib
	3	+
	4	+from msg.stream import accept_msg, write_msg
	5	+from msg.message import Message, OkMessage, KoMessage, NumMessage
	6	+
	7	+class UserInvalid(Exception):
	8	+ pass
	9	+
	10	+def validate_user(conns, pass_file):
	11	+ pass_dict = parse_passwd(pass_file)
	12	+ login = accept_msg(conns, Message).get_contents()
	13	+ passwd = accept_msg(conns, Message).get_contents()
	14	+ if pass_dict.has_key(login) and pass_dict[login] == passwd:
	15	+ write_msg(conns, OkMessage())
	16	+ return login
	17	+ else:
	18	+ write_msg(conns, KoMessage())
	19	+ raise UserInvalid(login)
	20	+
	21	+def parse_passwd(pass_file):
	22	+ result = {}
	23	+ with open(pass_file) as f:
	24	+ for line in f:
	25	+ login, passwd = line.split("=")
	26	+ login = login.strip()
	27	+ passwd = passwd.strip()
	28	+ result[login] = passwd
	29	+ return result
	30	+
	31	+def _check_sum(*args):
	32	+ m = hashlib.sha256()
	33	+ for arg in args:
	34	+ m.update(arg)
	35	+ return m
	36	+
	37	+def check_sum(*args):
	38	+ return _check_sum(*args).digest()
	39	+
	40	+def check_hexsum(*args):
	41	+ return _check_sum(*args).hexdigest()
	42	+
	43	+def send_contents(stream, contents):
	44	+ write_msg(stream, NumMessage(len(contents)))
	45	+ for ext, data in contents.iteritems():
	46	+ write_msg(stream, Message(ext))
	47	+ write_msg(stream, Message(data))
	48	+ write_msg(stream, Message(check_sum(ext, data)))
	49	+ accept_msg(stream, OkMessage)
	50	+
	51	+def receive_contents(stream):
	52	+ contents = {}
	53	+ n = accept_msg(stream, NumMessage).get_number()
	54	+ for _ in range(n):
	55	+ ext = accept_msg(stream, Message).get_contents()
	56	+ data = accept_msg(stream, Message).get_contents()
	57	+ csum = accept_msg(stream, Message).get_contents()
	58	+ if csum == check_sum(ext, data):
	59	+ write_msg(stream, OkMessage())
	60	+ else:
	61	+ write_msg(stream, KoMessage())
	62	+ contents[ext] = data
	63	+ return contents
...	...

example_configuration/passwd 0 → 100644

View file @44cbbee

	1	+++ a/example_configuration/passwd
	1	+adj1 = adj1
	2	+adj2 = adj2
	3	+ann1 = ann1
	4	+ann2 = ann2
	5	+ann3 = ann3
	6	+ann4 = ann4
...	...

example_configuration/server.cfg 0 → 100644

View file @44cbbee

	1	+++ a/example_configuration/server.cfg
	1	+file_extensions = .mmax _words.xml _mentions.xml
	2	+anno_per_file = 2
	3	+
	4	+[connection]
	5	+host = localhost
	6	+port = 2225
	7	+backlog = 5 # maximum number of waiting connections
	8	+{paths} # paths relative to configuration file
	9	+certfile = ssl.conf/cert.pem
	10	+keyfile = ssl.conf/key.pem
	11	+
	12	+
	13	+[svn]
	14	+login = machine
	15	+passwd = asdasdT7
	16	+{paths}
	17	+repository = ../data/repo_wc # path to working copy of repo
	18	+
	19	+
	20	+[users]
	21	+{paths}
	22	+config = users.cfg
	23	+passfile = passwd
...	...

example_configuration/ssl.conf/cert.pem 0 → 100644

View file @44cbbee

	1	+++ a/example_configuration/ssl.conf/cert.pem
	1	+-----BEGIN CERTIFICATE-----
	2	+MIICkDCCAfmgAwIBAgIJAJPcESSEV86nMA0GCSqGSIb3DQEBBQUAMGExCzAJBgNV
	3	+BAYTAlBMMQ0wCwYDVQQIDARUZXN0MQ0wCwYDVQQHDARDaXR5MRAwDgYDVQQKDAdD
	4	+b21wYW55MSIwIAYJKoZIhvcNAQkBFhNleGFtcGxlQGV4YW1wbGUuY29tMB4XDTEz
	5	+MDEyMzEyMjIwN1oXDTE0MDEyMzEyMjIwN1owYTELMAkGA1UEBhMCUEwxDTALBgNV
	6	+BAgMBFRlc3QxDTALBgNVBAcMBENpdHkxEDAOBgNVBAoMB0NvbXBhbnkxIjAgBgkq
	7	+hkiG9w0BCQEWE2V4YW1wbGVAZXhhbXBsZS5jb20wgZ8wDQYJKoZIhvcNAQEBBQAD
	8	+gY0AMIGJAoGBAKUG9QHjUiCSjcvFQl1OqUgZ8c4Vzm7m7Ua7D33YDnGo3QGuhPML
	9	+0JB+JPBG++MrPdQqMV8eeXe1dFlrm6SHnrEZIhwZg8iDnPhFKNjHiAkx0PCQdI93
	10	+rAe4EcszkgbQCXXLGjQJNDikwGZ6ciylJ7C2XewWTlWjv7/LqMwhQotLAgMBAAGj
	11	+UDBOMB0GA1UdDgQWBBRLRvOFIo9LIeiT2ohQalEzy687ijAfBgNVHSMEGDAWgBRL
	12	+RvOFIo9LIeiT2ohQalEzy687ijAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUA
	13	+A4GBAD1odtgH1kZ4a0TEbwjK2PaHTYyofkkgeXsGGnG3eMkwKQ7PHJarBl7GtxP5
	14	+Ls91k/KvBuNqCM/lPmM94USDSXrKGe0Zzc9WZbDG9SXO23gGOwPl+Tg9v1ocRHd2
	15	+dYYIgZM8/tLcsHyNNMd7c4BOMD4rzkjgbThFFMl/2NSkvrO7
	16	+-----END CERTIFICATE-----
...	...

example_configuration/ssl.conf/key.pem 0 → 100644

View file @44cbbee

	1	+++ a/example_configuration/ssl.conf/key.pem
	1	+-----BEGIN PRIVATE KEY-----
	2	+MIICeAIBADANBgkqhkiG9w0BAQEFAASCAmIwggJeAgEAAoGBAKUG9QHjUiCSjcvF
	3	+Ql1OqUgZ8c4Vzm7m7Ua7D33YDnGo3QGuhPML0JB+JPBG++MrPdQqMV8eeXe1dFlr
	4	+m6SHnrEZIhwZg8iDnPhFKNjHiAkx0PCQdI93rAe4EcszkgbQCXXLGjQJNDikwGZ6
	5	+ciylJ7C2XewWTlWjv7/LqMwhQotLAgMBAAECgYEAjx9XggmqgoFX5K/xfIbqHZQS
	6	+uE8FN/2jL0Kwjs3AySZQdlLrDvbiLU6ZrqGBd2VbPBRTuVDuu7ltbNf8plbQcxaa
	7	+yaKfKHPRYYIh/EqdaqsZ2bHHEARi03iNLDnktYz49I0ebFqgz0BGZVR3xDsjCu40
	8	+s9Tku7IQsDdO/zVvreECQQDXHW1vli7iuOmTYfwKqeIV7wFPj0vp8XrUVOKqjiG1
	9	+Fdf2ZlLrzeR/HbPbd2SL6u1dMV1j8wH/yPNRzFqNwOXpAkEAxGR5DzqrEE2ihyKL
	10	+WzQn6tTTA9FI8WPbgXdTm/R3dev2OgMLZycHlSgptCJ1f+fMfGNhWhotg8kZfrO0
	11	+9qnDEwJBAIpKrNQT5Mh0wBSz5WillmcrY9hV8yPOznw6pg2nmhvkkxYg6iYWE1N1
	12	+MO9ibg1VVouW9McQrrRV57cqfPysiiECQHP63d9Xh1R+dKAXj3LSD0zIWMGlY81i
	13	+amw/uvmb2ryiF+xUhfRqATld2ZsOypM5ofJHgmOCmCR+q3a+y/zrbk8CQQC0jx/T
	14	+TYw9ULoUe0bdx9RfJkrCEr7/+/DLs4MFwPiQ/e8foHcUL+CM9EY/FkMfbWbKjtuQ
	15	+iKXS/h5qp6NFdNVj
	16	+-----END PRIVATE KEY-----
...	...

example_configuration/users.cfg 0 → 100644

View file @44cbbee

	1	+++ a/example_configuration/users.cfg
	1	+[auth]
	2	+adjudicators = adj1 adj2
	3	+
	4	+[limits]
	5	+# number of files downloaded for annotation
	6	+annotation = 10
	7	+# number of file pairs download for adjudication
	8	+adjudication = 5
	9	+
	10	+# limits per user
	11	+# adj1.annotation = 2
	12	+# adj2.adjudication = 5
...	...

gui/CheckoutNumberDialog.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/CheckoutNumberDialog.py
	1	+# -- coding: utf-8 --
	2	+# generated by wxGlade 0.6.3 on Tue Sep 27 12:15:56 2011
	3	+
	4	+import wx
	5	+
	6	+# begin wxGlade: dependencies
	7	+# end wxGlade
	8	+
	9	+# begin wxGlade: extracode
	10	+
	11	+# end wxGlade
	12	+
	13	+"""aktualnie nieużywane! """
	14	+
	15	+class CheckoutNumberDialog(wx.Dialog):
	16	+ def __init__(self, args, *kwds):
	17	+ # begin wxGlade: CheckoutNumberDialog.__init__
	18	+ kwds["style"] = wx.DEFAULT_DIALOG_STYLE
	19	+ wx.Dialog.__init__(self, args, *kwds)
	20	+ self.label_4 = wx.StaticText(self, -1, "checkout files: ")
	21	+ self.checkout_number = wx.SpinCtrl(self, -1, "1", min=1, max=300)
	22	+ self.button_4 = wx.Button(self, wx.ID_YES, "OK")
	23	+ self.button_5 = wx.Button(self, wx.ID_NO, "Cancel")
	24	+
	25	+ self.__set_properties()
	26	+ self.__do_layout()
	27	+
	28	+ self.Bind(wx.EVT_BUTTON, self.on_ok, id=wx.ID_YES)
	29	+ self.Bind(wx.EVT_BUTTON, self.on_cancel, id=wx.ID_NO)
	30	+ # end wxGlade
	31	+
	32	+ def __set_properties(self):
	33	+ # begin wxGlade: CheckoutNumberDialog.__set_properties
	34	+ self.SetTitle("checkout")
	35	+ self.SetSize((193, 79))
	36	+ # end wxGlade
	37	+
	38	+ def __do_layout(self):
	39	+ # begin wxGlade: CheckoutNumberDialog.__do_layout
	40	+ sizer_8 = wx.BoxSizer(wx.VERTICAL)
	41	+ grid_sizer_3 = wx.GridSizer(1, 2, 0, 0)
	42	+ grid_sizer_2 = wx.GridSizer(1, 2, 0, 0)
	43	+ grid_sizer_2.Add(self.label_4, 0, wx.ALIGN_CENTER_VERTICAL, 0)
	44	+ grid_sizer_2.Add(self.checkout_number, 0, wx.ALIGN_CENTER_VERTICAL, 0)
	45	+ sizer_8.Add(grid_sizer_2, 1, 0, 0)
	46	+ grid_sizer_3.Add(self.button_4, 0, 0, 0)
	47	+ grid_sizer_3.Add(self.button_5, 0, 0, 0)
	48	+ sizer_8.Add(grid_sizer_3, 0, wx.BOTTOM, 0)
	49	+ self.SetSizer(sizer_8)
	50	+ self.Layout()
	51	+ self.Centre()
	52	+ # end wxGlade
	53	+
	54	+ def on_ok(self, event): # wxGlade: CheckoutNumberDialog.<event_handler>
	55	+ print "Event handler `on_ok' not implemented!"
	56	+ event.Skip()
	57	+
	58	+ def on_cancel(self, event): # wxGlade: CheckoutNumberDialog.<event_handler>
	59	+ print "Event handler `on_cancel' not implemented!"
	60	+ event.Skip()
	61	+
	62	+# end of class CheckoutNumberDialog
	63	+
	64	+
...	...

gui/ConfigDialog.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/ConfigDialog.py
	1	+# -- coding: utf-8 --
	2	+# generated by wxGlade 0.6.3 on Tue Sep 27 14:54:14 2011
	3	+
	4	+import wx
	5	+
	6	+import i18n
	7	+_ = i18n.language.ugettext #use ugettext instead of getttext to avoid unicode errors
	8	+
	9	+# begin wxGlade: dependencies
	10	+# end wxGlade
	11	+
	12	+# begin wxGlade: extracode
	13	+
	14	+# end wxGlade
	15	+
	16	+class ConfigDialog(wx.Dialog):
	17	+ def __init__(self, args, *kwds):
	18	+ # begin wxGlade: ConfigDialog.__init__
	19	+ kwds["style"] = wx.DEFAULT_DIALOG_STYLE
	20	+ wx.Dialog.__init__(self, args, *kwds)
	21	+ self.label_2 = wx.StaticText(self, -1, _("login")+":")
	22	+ self.login_ctrl = wx.TextCtrl(self, -1, "")
	23	+ self.label_6 = wx.StaticText(self, -1, _("password")+":")
	24	+ self.passwd_ctrl = wx.TextCtrl(self, -1, "", style=wx.TE_PASSWORD)
	25	+ self.label_5 = wx.StaticText(self, -1, _("extensions")+":")
	26	+ self.exts_ctrl = wx.TextCtrl(self, -1, "", size=(250,30))
	27	+ self.button_2 = wx.Button(self, wx.ID_OK, _("OK"))
	28	+
	29	+ self.__set_properties()
	30	+ self.__do_layout()
	31	+ # end wxGlade
	32	+
	33	+ def __set_properties(self):
	34	+ # begin wxGlade: ConfigDialog.__set_properties
	35	+ self.SetTitle(_("Directory configuration"))
	36	+ self.label_2.SetToolTipString(_("Login supplied by system administrator"))
	37	+ self.label_6.SetToolTipString(_("Password supplied by system administrator"))
	38	+ self.label_5.SetToolTipString(_("List of space separated file extensions"))
	39	+ self.button_2.SetMinSize((85, 29))
	40	+ self.button_2.SetFocus()
	41	+ # end wxGlade
	42	+
	43	+ def __do_layout(self):
	44	+ # begin wxGlade: ConfigDialog.__do_layout
	45	+ sizer_5 = wx.BoxSizer(wx.VERTICAL)
	46	+ grid_sizer_4 = wx.GridSizer(3, 2, 0, 0)
	47	+ grid_sizer_4.Add(self.label_2, 0, wx.ALIGN_CENTER_VERTICAL, 0)
	48	+ grid_sizer_4.Add(self.login_ctrl, 0, wx.ALIGN_CENTER_VERTICAL, 0)
	49	+ grid_sizer_4.Add(self.label_6, 0, wx.ALIGN_CENTER_VERTICAL, 0)
	50	+ grid_sizer_4.Add(self.passwd_ctrl, 0, wx.ALIGN_CENTER_VERTICAL, 0)
	51	+ grid_sizer_4.Add(self.label_5, 0, wx.ALIGN_CENTER_VERTICAL, 0)
	52	+ grid_sizer_4.Add(self.exts_ctrl, 0, wx.ALIGN_CENTER_VERTICAL, 0)
	53	+ sizer_5.Add(grid_sizer_4, 0, 0, 0)
	54	+ sizer_5.Add(self.button_2, 0, wx.ALIGN_CENTER_HORIZONTAL, 0)
	55	+ self.SetSizer(sizer_5)
	56	+ sizer_5.Fit(self)
	57	+ self.Layout()
	58	+ # end wxGlade
	59	+
	60	+# end of class ConfigDialog
	61	+
	62	+
...	...

gui/DownloadNumberDialog.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/DownloadNumberDialog.py
	1	+# -- coding: utf-8 --
	2	+# generated by wxGlade 0.6.3 on Wed Sep 28 10:10:39 2011
	3	+
	4	+import wx
	5	+
	6	+import i18n
	7	+_ = i18n.language.ugettext #use ugettext instead of getttext to avoid unicode errors
	8	+
	9	+# begin wxGlade: dependencies
	10	+# end wxGlade
	11	+
	12	+# begin wxGlade: extracode
	13	+
	14	+# end wxGlade
	15	+
	16	+class DownloadNumberDialog(wx.Dialog):
	17	+ def __init__(self, args, *kwds):
	18	+ # begin wxGlade: DownloadNumberDialog.__init__
	19	+ kwds["style"] = wx.DEFAULT_DIALOG_STYLE
	20	+ wx.Dialog.__init__(self, args, *kwds)
	21	+ self.label_4 = wx.StaticText(self, -1, _("Download count")+":")
	22	+ self.checkout_number = wx.SpinCtrl(self, -1, "1", min=1, max=300)
	23	+ self.button_4 = wx.Button(self, wx.ID_YES, _("OK"))
	24	+ self.button_5 = wx.Button(self, wx.ID_NO, _("Cancel"))
	25	+
	26	+ self.__set_properties()
	27	+ self.__do_layout()
	28	+
	29	+ self.Bind(wx.EVT_BUTTON, self.on_ok, id=wx.ID_YES)
	30	+ self.Bind(wx.EVT_BUTTON, self.on_cancel, id=wx.ID_NO)
	31	+ # end wxGlade
	32	+
	33	+ def __set_properties(self):
	34	+ # begin wxGlade: DownloadNumberDialog.__set_properties
	35	+ self.SetTitle(_("Download"))
	36	+ self.label_4.SetToolTipString(_("Number of files to download"))
	37	+ # end wxGlade
	38	+
	39	+ def __do_layout(self):
	40	+ # begin wxGlade: DownloadNumberDialog.__do_layout
	41	+ grid_sizer_2 = wx.GridSizer(2, 2, 20, 20)
	42	+ grid_sizer_2.Add(self.label_4, 0, wx.ALIGN_CENTER_HORIZONTAL\|wx.ALIGN_CENTER_VERTICAL, 0)
	43	+ grid_sizer_2.Add(self.checkout_number, 0, 0, 0)
	44	+ grid_sizer_2.Add(self.button_4, 0, wx.ALIGN_CENTER_HORIZONTAL\|wx.ALIGN_CENTER_VERTICAL, 0)
	45	+ grid_sizer_2.Add(self.button_5, 0, wx.ALIGN_CENTER_HORIZONTAL\|wx.ALIGN_CENTER_VERTICAL, 0)
	46	+ self.SetSizer(grid_sizer_2)
	47	+ grid_sizer_2.Fit(self)
	48	+ self.Layout()
	49	+ self.Centre()
	50	+ # end wxGlade
	51	+
	52	+ def on_ok(self, event): # wxGlade: DownloadNumberDialog.<event_handler>
	53	+ self.EndModal(wx.ID_OK)
	54	+
	55	+ def on_cancel(self, event): # wxGlade: DownloadNumberDialog.<event_handler>
	56	+ self.EndModal(wx.ID_CANCEL)
	57	+
	58	+# end of class DownloadNumberDialog
	59	+
	60	+
...	...

gui/LogDialog.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/LogDialog.py
	1	+# -- coding: utf-8 --
	2	+# generated by wxGlade 0.6.3 on Tue Sep 27 12:15:56 2011
	3	+
	4	+import wx
	5	+
	6	+# begin wxGlade: dependencies
	7	+# end wxGlade
	8	+
	9	+# begin wxGlade: extracode
	10	+
	11	+# end wxGlade
	12	+
	13	+"""aktualnie nieużywane! """
	14	+
	15	+class LogDialog(wx.Dialog):
	16	+ def __init__(self, args, *kwds):
	17	+ # begin wxGlade: LogDialog.__init__
	18	+ kwds["style"] = wx.DEFAULT_DIALOG_STYLE\|wx.RESIZE_BORDER\|wx.THICK_FRAME
	19	+ wx.Dialog.__init__(self, args, *kwds)
	20	+
	21	+ self.__set_properties()
	22	+ self.__do_layout()
	23	+ # end wxGlade
	24	+
	25	+ def __set_properties(self):
	26	+ # begin wxGlade: LogDialog.__set_properties
	27	+ self.SetTitle("Log")
	28	+ self.SetSize((400, 315))
	29	+ # end wxGlade
	30	+
	31	+ def __do_layout(self):
	32	+ # begin wxGlade: LogDialog.__do_layout
	33	+ self.Layout()
	34	+ # end wxGlade
	35	+
	36	+ def on_ok(self, event): # wxGlade: LogDialog.<event_handler>
	37	+ print "Event handler `on_ok' not implemented!"
	38	+ event.Skip()
	39	+
	40	+# end of class LogDialog
	41	+
	42	+
...	...

gui/LoginDialog.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/LoginDialog.py
	1	+# -- coding: utf-8 --
	2	+# generated by wxGlade 0.6.3 on Tue Sep 27 12:15:56 2011
	3	+
	4	+import wx
	5	+
	6	+# begin wxGlade: dependencies
	7	+# end wxGlade
	8	+
	9	+# begin wxGlade: extracode
	10	+
	11	+# end wxGlade
	12	+
	13	+"""aktualnie nieużywane! """
	14	+
	15	+class LoginDialog(wx.Dialog):
	16	+ def __init__(self, args, *kwds):
	17	+ # begin wxGlade: LoginDialog.__init__
	18	+ kwds["style"] = wx.DEFAULT_DIALOG_STYLE\|wx.RESIZE_BORDER\|wx.THICK_FRAME
	19	+ wx.Dialog.__init__(self, args, *kwds)
	20	+ self.label_1 = wx.StaticText(self, -1, "login: ")
	21	+ self.login_ctrl = wx.TextCtrl(self, -1, "")
	22	+ self.label_3 = wx.StaticText(self, -1, "password: ")
	23	+ self.passwd_ctrl = wx.TextCtrl(self, -1, "", style=wx.TE_PASSWORD)
	24	+ self.save_passwd = wx.CheckBox(self, -1, "save password")
	25	+ self.button_1 = wx.Button(self, wx.ID_OK, "OK")
	26	+
	27	+ self.__set_properties()
	28	+ self.__do_layout()
	29	+
	30	+ self.Bind(wx.EVT_BUTTON, self.on_ok, id=wx.ID_OK)
	31	+ # end wxGlade
	32	+
	33	+ def __set_properties(self):
	34	+ # begin wxGlade: LoginDialog.__set_properties
	35	+ self.SetTitle("login")
	36	+ self.button_1.SetMinSize((85, 32))
	37	+ # end wxGlade
	38	+
	39	+ def __do_layout(self):
	40	+ # begin wxGlade: LoginDialog.__do_layout
	41	+ sizer_4 = wx.BoxSizer(wx.VERTICAL)
	42	+ grid_sizer_1 = wx.GridSizer(2, 2, 0, 0)
	43	+ grid_sizer_1.Add(self.label_1, 0, wx.ALIGN_CENTER_VERTICAL, 0)
	44	+ grid_sizer_1.Add(self.login_ctrl, 0, 0, 0)
	45	+ grid_sizer_1.Add(self.label_3, 0, wx.ALIGN_CENTER_VERTICAL, 0)
	46	+ grid_sizer_1.Add(self.passwd_ctrl, 0, 0, 0)
	47	+ sizer_4.Add(grid_sizer_1, 0, 0, 0)
	48	+ sizer_4.Add(self.save_passwd, 0, 0, 0)
	49	+ sizer_4.Add(self.button_1, 0, wx.ALIGN_CENTER_HORIZONTAL, 0)
	50	+ self.SetSizer(sizer_4)
	51	+ sizer_4.Fit(self)
	52	+ self.Layout()
	53	+ # end wxGlade
	54	+
	55	+ def on_ok(self, event): # wxGlade: LoginDialog.<event_handler>
	56	+ print "Event handler `on_ok' not implemented!"
	57	+ event.Skip()
	58	+
	59	+# end of class LoginDialog
	60	+
	61	+
...	...

gui/MainFrame.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/MainFrame.py
	1	+# -- coding: utf-8 --
	2	+# generated by wxGlade 0.6.3 on Tue Sep 27 12:15:56 2011
	3	+
	4	+import wx
	5	+
	6	+import i18n
	7	+_ = i18n.language.ugettext
	8	+
	9	+from DownloadNumberDialog import DownloadNumberDialog
	10	+from ReturnReasonDialog import ReturnReasonDialog
	11	+from SortableListCtrl import SortableListCtrl
	12	+
	13	+import const
	14	+import utils
	15	+
	16	+# begin wxGlade: dependencies
	17	+# end wxGlade
	18	+
	19	+# begin wxGlade: extracode
	20	+
	21	+# end wxGlade
	22	+
	23	+class MainFrame(wx.Frame):
	24	+ def __init__(self, args, *kwds):
	25	+ # begin wxGlade: MainFrame.__init__
	26	+ kwds["style"] = wx.DEFAULT_FRAME_STYLE
	27	+ wx.Frame.__init__(self, args, *kwds)
	28	+ self.window_1 = wx.SplitterWindow(self, -1, style=wx.SP_3D\|wx.SP_BORDER)
	29	+ self.window_1_pane_2 = wx.Panel(self.window_1, -1)
	30	+ self.window_1_pane_1 = wx.Panel(self.window_1, -1)
	31	+ self.notebook_1 = wx.Notebook(self.window_1_pane_1, -1, style=wx.BK_DEFAULT)
	32	+ self.notebook_1_pane_1 = wx.Panel(self.notebook_1, -1)
	33	+ self.notebook_1_pane_2 = wx.Panel(self.notebook_1, -1)
	34	+
	35	+ self.log_ctrl = wx.TextCtrl(self.window_1_pane_2, -1, "", style=wx.TE_MULTILINE\|wx.TE_READONLY)
	36	+
	37	+ self.files_list = SortableListCtrl(self.notebook_1_pane_1, self.log_ctrl, False, const.program, const.extension)
	38	+ self.super_files_list = SortableListCtrl(self.notebook_1_pane_2, self.log_ctrl, True, const.program, const.extension)
	39	+
	40	+ # Menu Bar
	41	+ self.frame_1_menubar = wx.MenuBar()
	42	+ self.dir_menu = wx.Menu()
	43	+ self.dir_menu_open = wx.MenuItem(self.dir_menu, 5, _("Open"), _("Open new directory"), wx.ITEM_NORMAL)
	44	+ self.dir_menu.AppendItem(self.dir_menu_open)
	45	+ self.dir_menu.Append(6, _("Reconfigure"), _("Reconfigure directory"), wx.ITEM_NORMAL)
	46	+ self.dir_menu.AppendSeparator()
	47	+ if not const.hide_dir_menu == "yes":
	48	+ self.frame_1_menubar.Append(self.dir_menu, _("Directory"))
	49	+ wxglade_tmp_menu = wx.Menu()
	50	+ wxglade_tmp_menu.Append(1, _("Upload Selected"), _("Upload annotated or adjudicated files"), wx.ITEM_NORMAL)
	51	+ wxglade_tmp_menu.Append(8, _("Checkin Selected"), _("Upload annotated or adjudicated; files will be deleted from local disk and you will not be able to modify them again"), wx.ITEM_NORMAL)
	52	+ wxglade_tmp_menu.Append(10, _("Return Selected"), _("Return annotated or adjudicated because they are not suitable for annotation or contain errors"), wx.ITEM_NORMAL)
	53	+ wxglade_tmp_menu.AppendSeparator()
	54	+ wxglade_tmp_menu.Append(2, _("Download (annotation)"), _("Download files for annotation"), wx.ITEM_NORMAL)
	55	+ wxglade_tmp_menu.Append(3, _("Download (adjudication)"), _("Download files for adjudication"), wx.ITEM_NORMAL)
	56	+ wxglade_tmp_menu.AppendSeparator()
	57	+ wxglade_tmp_menu.Append(7, _("Checkout"), _("Checkout files from server to your local, empty directory"), wx.ITEM_NORMAL)
	58	+
	59	+ self.frame_1_menubar.Append(wxglade_tmp_menu, _("Files"))
	60	+ wxglade_tmp_menu = wx.Menu()
	61	+ wxglade_tmp_menu.Append(4, _("Ready"), _("Select items which are marked for upload"), wx.ITEM_NORMAL)
	62	+ wxglade_tmp_menu.Append(9, _("Modified"), _("Select items which are modified"), wx.ITEM_NORMAL)
	63	+ self.frame_1_menubar.Append(wxglade_tmp_menu, _("Select"))
	64	+
	65	+ wxglade_tmp_menu = wx.Menu()
	66	+ wxglade_tmp_menu.Append(11, _("Finished count"), _("Check number of finished texts"), wx.ITEM_NORMAL)
	67	+ self.frame_1_menubar.Append(wxglade_tmp_menu, _("Annotation"))
	68	+
	69	+ self.SetMenuBar(self.frame_1_menubar)
	70	+ # Menu Bar end
	71	+
	72	+ self.CreateStatusBar()
	73	+
	74	+ self.__set_properties()
	75	+ self.__do_layout()
	76	+
	77	+ self.Bind(wx.EVT_MENU, self.open_directory, self.dir_menu_open)
	78	+ self.Bind(wx.EVT_MENU, self.reconf_directory, id=6)
	79	+ self.Bind(wx.EVT_MENU, self.upload, id=1)
	80	+ self.Bind(wx.EVT_MENU, self.checkin, id=8)
	81	+ self.Bind(wx.EVT_MENU, self.download, id=2)
	82	+ self.Bind(wx.EVT_MENU, self.download_prim, id=3)
	83	+ self.Bind(wx.EVT_MENU, self.checkout, id=7)
	84	+ self.Bind(wx.EVT_MENU, self.select_ready, id=4)
	85	+ self.Bind(wx.EVT_MENU, self.select_modified, id=9)
	86	+ self.Bind(wx.EVT_MENU, self.return_files, id=10)
	87	+ self.Bind(wx.EVT_MENU, self.check_stats, id=11)
	88	+
	89	+ self.Bind(wx.EVT_LIST_ITEM_ACTIVATED, self.activate_item, self.files_list)
	90	+ self.Bind(wx.EVT_LIST_ITEM_ACTIVATED, self.activate_super_item, self.super_files_list)
	91	+
	92	+ def __set_properties(self):
	93	+ # begin wxGlade: MainFrame.__set_properties
	94	+ self.SetTitle("--")
	95	+ self.SetSize((663, 556))
	96	+ self.SetFocus()
	97	+ # end wxGlade
	98	+
	99	+ def __do_layout(self):
	100	+ # begin wxGlade: MainFrame.__do_layout
	101	+ sizer_6 = wx.BoxSizer(wx.HORIZONTAL)
	102	+ sizer_7 = wx.BoxSizer(wx.VERTICAL)
	103	+ sizer_1 = wx.BoxSizer(wx.VERTICAL)
	104	+ sizer_3 = wx.BoxSizer(wx.VERTICAL)
	105	+ sizer_2 = wx.BoxSizer(wx.VERTICAL)
	106	+ sizer_2.Add(self.files_list, 1, wx.EXPAND, 0)
	107	+ self.notebook_1_pane_1.SetSizer(sizer_2)
	108	+ sizer_3.Add(self.super_files_list, 1, wx.EXPAND, 0)
	109	+ self.notebook_1_pane_2.SetSizer(sizer_3)
	110	+ self.notebook_1.AddPage(self.notebook_1_pane_1, _("Annotation"))
	111	+ self.notebook_1.AddPage(self.notebook_1_pane_2, _("Adjudication"))
	112	+ sizer_1.Add(self.notebook_1, 1, wx.EXPAND, 0)
	113	+ self.window_1_pane_1.SetSizer(sizer_1)
	114	+ sizer_7.Add(self.log_ctrl, 1, wx.EXPAND, 0)
	115	+ self.window_1_pane_2.SetSizer(sizer_7)
	116	+ self.window_1.SplitHorizontally(self.window_1_pane_1, self.window_1_pane_2, -100)
	117	+ sizer_6.Add(self.window_1, 1, wx.EXPAND, 0)
	118	+ self.SetSizer(sizer_6)
	119	+ self.Layout()
	120	+ self.Centre()
	121	+ # end wxGlade
	122	+
	123	+ def set_manager(self, manager):
	124	+ self.manager = manager
	125	+
	126	+ def open_directory(self, event): # wxGlade: MainFrame.<event_handler>
	127	+ if event.GetId() == self.dir_menu_open.GetId():
	128	+ dlg = wx.DirDialog(self, message=_("Choose a directory"))
	129	+ try:
	130	+ if dlg.ShowModal() == wx.ID_OK:
	131	+ self.manager.set_directory(dlg.GetPath())
	132	+ finally:
	133	+ dlg.Destroy()
	134	+ else:
	135	+ item = self.dir_menu.FindItemById(event.GetId())
	136	+ self.manager.set_directory(item.GetLabel())
	137	+
	138	+ def upload(self, event): # wxGlade: MainFrame.<event_handler>
	139	+ names = ( self.files_list.get_selected()
	140	+ + self.super_files_list.get_selected() )
	141	+ info = ( _("Do you want to upload the following files?")+"\n\n"
	142	+ + "\n".join(names) )
	143	+ dial = wx.MessageDialog( self, info, _('Question')
	144	+ , wx.YES_NO \| wx.NO_DEFAULT \| wx.ICON_QUESTION )
	145	+ try:
	146	+ if dial.ShowModal() == wx.ID_YES:
	147	+ self.manager.upload(names, checkin=False)
	148	+ finally:
	149	+ dial.Destroy()
	150	+
	151	+ def return_files(self, event): # wxGlade: MainFrame.<event_handler>
	152	+ names = ( self.files_list.get_selected()
	153	+ + self.super_files_list.get_selected() )
	154	+ reason = None
	155	+ dialog = ReturnReasonDialog(self, names)
	156	+ try:
	157	+ if dialog.ShowModal() == wx.ID_OK:
	158	+ reason = dialog.reason.GetValue().encode("utf-8")
	159	+ finally:
	160	+ dialog.Destroy()
	161	+ if reason is not None:
	162	+ self.manager.return_files(names, reason)
	163	+
	164	+ def download(self, event): # wxGlade: MainFrame.<event_handler>
	165	+ num = None
	166	+ dialog = DownloadNumberDialog(self, -1)
	167	+ try:
	168	+ if dialog.ShowModal() == wx.ID_OK:
	169	+ num = dialog.checkout_number.GetValue()
	170	+ finally:
	171	+ dialog.Destroy()
	172	+ if num is not None:
	173	+ self.manager.download(num)
	174	+
	175	+ def download_prim(self, event): # wxGlade: MainFrame.<event_handler>
	176	+ num = None
	177	+ dialog = DownloadNumberDialog(self, -1)
	178	+ try:
	179	+ if dialog.ShowModal() == wx.ID_OK:
	180	+ num = dialog.checkout_number.GetValue()
	181	+ finally:
	182	+ dialog.Destroy()
	183	+ if num is not None:
	184	+ self.manager.download_prim(num)
	185	+
	186	+ def activate_item(self, event): # wxGlade: MainFrame.<event_handler>
	187	+ utils.change_ready( self.manager.ready
	188	+ , self.files_list
	189	+ , event.GetIndex() )
	190	+
	191	+ def activate_super_item(self, event): # wxGlade: MainFrame.<event_handler>
	192	+ utils.change_ready_prim( self.manager.ready
	193	+ , self.super_files_list
	194	+ , event.GetIndex() )
	195	+
	196	+ def reconf_directory(self, event): # wxGlade: MainFrame.<event_handler>
	197	+ self.manager.reconf_directory()
	198	+
	199	+ def select_ready(self, event): # wxGlade: MainFrame.<event_handler>
	200	+ self.files_list.select_ready()
	201	+ self.super_files_list.select_ready()
	202	+
	203	+ def checkout(self, event): # wxGlade: MainFrame.<event_handler>
	204	+ self.manager.checkout()
	205	+
	206	+ def checkin(self, event): # wxGlade: MainFrame.<event_handler>
	207	+ names = ( self.files_list.get_selected()
	208	+ + self.super_files_list.get_selected() )
	209	+ info = ( _("Do you want to checkin the following files?")+"\n\n"
	210	+ + "\n".join(names)
	211	+ + "\n\n"+_("Files will be deleted from the local directory and you will not be able to read, write or modify them later."))
	212	+ dial = wx.MessageDialog( self, info, _('Question')
	213	+ , wx.YES_NO \| wx.NO_DEFAULT \| wx.ICON_QUESTION )
	214	+ try:
	215	+ if dial.ShowModal() == wx.ID_YES:
	216	+ self.manager.upload(names, checkin=True)
	217	+ finally:
	218	+ dial.Destroy()
	219	+
	220	+ def select_modified(self, event): # wxGlade: MainFrame.<event_handler>
	221	+ self.files_list.select_modified()
	222	+ self.super_files_list.select_modified()
	223	+
	224	+ def check_stats(self, event): # wxGlade: MainFrame.<event_handler>
	225	+ self.manager.check_stats()
	226	+
	227	+# end of class MainFrame
...	...

gui/ReturnReasonDialog.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/ReturnReasonDialog.py
	1	+# -- coding: utf-8 --
	2	+# generated by wxGlade 0.6.3 on Wed Sep 28 10:10:39 2011
	3	+
	4	+import wx
	5	+
	6	+import i18n
	7	+_ = i18n.language.ugettext #use ugettext instead of getttext to avoid unicode errors
	8	+
	9	+# begin wxGlade: dependencies
	10	+# end wxGlade
	11	+
	12	+# begin wxGlade: extracode
	13	+
	14	+# end wxGlade
	15	+
	16	+class ReturnReasonDialog(wx.Dialog):
	17	+ def __init__(self, parent, names):
	18	+ # begin wxGlade: ReturnReasonDialog.__init__
	19	+ wx.Dialog.__init__(self, parent, -1)
	20	+
	21	+ info = ( _("Why do you want to return the following files?")+"\n"
	22	+ + "\n".join(names) )
	23	+
	24	+ self.label_1 = wx.StaticText(self, -1, info)
	25	+ self.label_4 = wx.StaticText(self, -1, _("Reason")+":")
	26	+ self.reason = wx.TextCtrl(self, -1, "", size=(250, 80), style=wx.TE_MULTILINE\|wx.TE_PROCESS_ENTER)
	27	+ self.button_4 = wx.Button(self, wx.ID_YES, _("OK"))
	28	+ self.button_5 = wx.Button(self, wx.ID_NO, _("Cancel"))
	29	+
	30	+ self.__set_properties()
	31	+ self.__do_layout()
	32	+
	33	+ self.Bind(wx.EVT_BUTTON, self.on_ok, id=wx.ID_YES)
	34	+ self.Bind(wx.EVT_BUTTON, self.on_cancel, id=wx.ID_NO)
	35	+ # end wxGlade
	36	+
	37	+ def __set_properties(self):
	38	+ # begin wxGlade: ReturnReasonDialog.__set_properties
	39	+ self.SetTitle(_("Return files"))
	40	+ # end wxGlade
	41	+
	42	+ def __do_layout(self):
	43	+ # begin wxGlade: ReturnReasonDialog.__do_layout
	44	+ sizer_2 = wx.BoxSizer(wx.HORIZONTAL)
	45	+ sizer_2.Add(self.label_4, 0, wx.ALIGN_CENTER_HORIZONTAL\|wx.ALIGN_CENTER_VERTICAL, 0)
	46	+ sizer_2.Add(self.reason, 0, 0, 0)
	47	+
	48	+ sizer_3 = wx.BoxSizer(wx.HORIZONTAL)
	49	+ sizer_3.Add(self.button_4, 0, wx.ALIGN_CENTER_HORIZONTAL\|wx.ALIGN_CENTER_VERTICAL, 0)
	50	+ sizer_3.Add(self.button_5, 0, wx.ALIGN_CENTER_HORIZONTAL\|wx.ALIGN_CENTER_VERTICAL, 0)
	51	+
	52	+ sizer_1 = wx.BoxSizer(wx.VERTICAL)
	53	+ sizer_1.Add(self.label_1, 0, wx.ALIGN_CENTER_HORIZONTAL\|wx.ALIGN_CENTER_VERTICAL, 1)
	54	+ sizer_1.Add(sizer_2, 0, wx.ALIGN_CENTER_HORIZONTAL\|wx.ALIGN_CENTER_VERTICAL, 1)
	55	+ sizer_1.Add(sizer_3, 0, wx.ALIGN_CENTER_HORIZONTAL\|wx.ALIGN_CENTER_VERTICAL, 1)
	56	+ self.SetSizer(sizer_1)
	57	+ sizer_1.Fit(self)
	58	+ self.Layout()
	59	+ # end wxGlade
	60	+
	61	+ def on_ok(self, event): # wxGlade: ReturnReasonDialog.<event_handler>
	62	+ self.EndModal(wx.ID_OK)
	63	+
	64	+ def on_cancel(self, event): # wxGlade: ReturnReasonDialog.<event_handler>
	65	+ self.EndModal(wx.ID_CANCEL)
	66	+
	67	+# end of class ReturnReasonDialog
	68	+
	69	+
...	...

gui/SortableListCtrl.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/SortableListCtrl.py
	1	+# -- coding: utf-8 --
	2	+import threading
	3	+from PIL import Image
	4	+
	5	+import i18n
	6	+_ = i18n.language.ugettext #use ugettext instead of getttext to avoid unicode errors
	7	+
	8	+import os.path
	9	+import wx
	10	+import wx.lib.mixins.listctrl as listmix
	11	+import subprocess
	12	+
	13	+try:
	14	+ from agw import ultimatelistctrl as ULC
	15	+except ImportError: # if it's not there locally, try the wxPython lib.
	16	+ from wx.lib.agw import ultimatelistctrl as ULC
	17	+
	18	+class SortableListCtrl(ULC.UltimateListCtrl, listmix.ColumnSorterMixin, listmix.ListCtrlAutoWidthMixin):
	19	+ def __init__(self, parent, log, superann, program, extension):
	20	+
	21	+ self._lock = threading.Lock()
	22	+
	23	+ self.program = program
	24	+ self.extension = extension
	25	+
	26	+ ULC.UltimateListCtrl.__init__(self, parent, -1, agwStyle=ULC.ULC_REPORT \| ULC.ULC_HAS_VARIABLE_ROW_HEIGHT \| ULC.ULC_USER_ROW_HEIGHT)#\| wx.LC_HRULES \| wx.LC_VRULES )
	27	+ listmix.ListCtrlAutoWidthMixin.__init__(self)
	28	+
	29	+ self.log_ctrl = log
	30	+
	31	+ self.il = wx.ImageList(16, 16)
	32	+ up = wx.Image(os.path.join('img', 'up.png'), wx.BITMAP_TYPE_PNG).ConvertToBitmap()
	33	+ down = wx.Image(os.path.join('img', 'down.png'), wx.BITMAP_TYPE_PNG).ConvertToBitmap()
	34	+ if superann:
	35	+ textimg_1 = wx.Image(os.path.join('img', 'textimg_2.png'), wx.BITMAP_TYPE_PNG).ConvertToBitmap()
	36	+ else:
	37	+ textimg_1 = wx.Image(os.path.join('img', 'textimg_1.png'), wx.BITMAP_TYPE_PNG).ConvertToBitmap()
	38	+ ready = wx.Image(os.path.join('img', 'ready.png'), wx.BITMAP_TYPE_PNG).ConvertToBitmap()
	39	+ self.sm_up = self.il.Add(up)
	40	+ self.sm_dn = self.il.Add(down)
	41	+ self.textimg_1 = self.il.Add(textimg_1)
	42	+ self.ready = self.il.Add(ready)
	43	+
	44	+ self.AssignImageList(self.il, wx.IMAGE_LIST_SMALL)
	45	+
	46	+ self.SetUserLineHeight(23)
	47	+ self.InsertColumn(0, _("File"))
	48	+ self.InsertColumn(1, _("Status"))
	49	+ self.InsertColumn(2, _("Ready"))
	50	+ self.InsertColumn(3, _("Stage"))
	51	+ self.InsertColumn(4, "")
	52	+ self.SetColumnWidth(0, 80)
	53	+ self.SetColumnWidth(1, 100)
	54	+ self.SetColumnWidth(2, 120)
	55	+ self.SetColumnWidth(3, 100)
	56	+
	57	+ self.itemDataMap = {}
	58	+
	59	+ listmix.ColumnSorterMixin.__init__(self, 5)
	60	+
	61	+ # required by ColumnSorterMixin
	62	+ def GetListCtrl(self):
	63	+ return self
	64	+
	65	+ def GetSortImages(self):
	66	+ return (self.sm_dn, self.sm_up)
	67	+ # required by ColumnSorterMixin - end
	68	+
	69	+ def show(self, data):
	70	+ self._lock.acquire()
	71	+ try:
	72	+ # for some reason deleteAllItems() doesn't work with buttons inside ulc
	73	+ for item in range(self.GetItemCount()):
	74	+ self.DeleteItem(0)
	75	+ self.itemDataMap.clear()
	76	+ for i, (file_name, status, is_ready, stage) in enumerate(data):
	77	+ img = self.textimg_1
	78	+ if is_ready == _("Yes"):
	79	+ img = self.ready
	80	+ pos = self.InsertImageStringItem(i, file_name, img)
	81	+ self.SetStringItem(pos, 1, status)
	82	+ self.SetStringItem(pos, 2, is_ready)
	83	+ self.SetStringItem(pos, 3, stage)
	84	+
	85	+ button = wx.Button(self, id=i, size=(80, 23), label=_("Open"))
	86	+ self.SetItemWindow(pos, col=4, wnd=button, expand=False)
	87	+ button.Bind(wx.EVT_BUTTON, self.open_button)
	88	+
	89	+ self.SetItemData(pos, i)
	90	+
	91	+ self.itemDataMap[i] = (file_name, status, is_ready, stage, None)
	92	+ finally:
	93	+ self._lock.release()
	94	+
	95	+ def shown_data(self):
	96	+ result = []
	97	+ for i in range(self.GetItemCount()):
	98	+ part_result = []
	99	+ for j in range(0, 4):
	100	+ part_result.append(self.GetItem(i, j).GetText())
	101	+ result.append(tuple(part_result))
	102	+ return result
	103	+
	104	+ def SetReadyYes(self, pos):
	105	+ self.SetItemImage(pos, self.ready)
	106	+ return self._set_ready(pos, _("Yes"))
	107	+
	108	+ def SetReadyNo(self, pos):
	109	+ self.SetItemImage(pos, self.textimg_1)
	110	+ return self._set_ready(pos, _("No"))
	111	+
	112	+ def _set_ready(self, pos, val):
	113	+ file_name = self.GetItem(pos, 0).GetText()
	114	+ for key, (filename, status, ready, stage, sth) in self.itemDataMap.iteritems():
	115	+ if filename == file_name:
	116	+ self.itemDataMap[key] = (filename, status, val, stage, sth)
	117	+ break
	118	+ return self.SetStringItem(pos, 2, val)
	119	+
	120	+ def GetFilename(self, pos):
	121	+ return self.GetItem(pos, 0).GetText()
	122	+
	123	+ def IsReady(self, pos):
	124	+ return self.GetItem(pos, 2).GetText() == _("Yes")
	125	+
	126	+ def select_ready(self):
	127	+ return self._select(lambda list, pos : list.GetItem(pos, 2).GetText() == _("Yes"))
	128	+
	129	+ def select_modified(self):
	130	+ return self._select(lambda list, pos : list.GetItem(pos, 1).GetText() == _("Modified"))
	131	+
	132	+ def _select(self, cond):
	133	+ for i in range(self.GetItemCount()):
	134	+ if cond(self, i):
	135	+ self.Select(i, 1)
	136	+ else:
	137	+ self.Select(i, 0)
	138	+
	139	+ def get_selected(self):
	140	+ fnames = []
	141	+ for i in range(self.GetItemCount()):
	142	+ if self.IsSelected(i):
	143	+ fnames.append(self.GetFilename(i))
	144	+ return fnames
	145	+
	146	+ def open_button(self, event):
	147	+ file_name = self.GetFilename(event.GetEventObject().GetId())
	148	+ print event.GetEventObject().GetId()
	149	+ if file_name:
	150	+ self.log_ctrl.AppendText(_("Starting ") + self.program + _(" for text") + " " + file_name + "...\n")
	151	+ app = os.path.join("..", self.program, self.program+".exe")
	152	+ app_bin = os.path.join("..", self.program, self.program)
	153	+ if not os.path.exists(app):
	154	+ app = app_bin
	155	+ text = os.path.join("..", "teksty", file_name + self.extension)
	156	+ try:
	157	+ subprocess.Popen([app, text])
	158	+ except Exception as ex:
	159	+ self.log_ctrl.AppendText(_("Error starting ") + self.program + ": " + unicode(str(ex), errors='replace') + "\n")
	160	+
...	...

gui/client.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/client.py
	1	+# -- coding: utf-8 --
	2	+import sys
	3	+import os
	4	+import shutil
	5	+import tempfile
	6	+import socket
	7	+import ssl
	8	+import pprint
	9	+from optparse import OptionParser
	10	+import traceback
	11	+import re
	12	+
	13	+import i18n
	14	+_ = i18n.language.ugettext
	15	+
	16	+from dfs.msg.stream import write_msg, read_msg, accept_msg
	17	+from dfs.msg.message import *
	18	+from dfs.utils import check_sum, send_contents, receive_contents
	19	+
	20	+from utils import normal_files, super_files
	21	+
	22	+def init_stream(host, port, cert, log=None):
	23	+ log_msg(_("Connecting server..."), log)
	24	+ s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
	25	+ stream = ssl.wrap_socket(s, ca_certs=cert, cert_reqs=ssl.CERT_REQUIRED, ssl_version=ssl.PROTOCOL_TLSv1)
	26	+ stream.connect((host, port))
	27	+ return stream
	28	+
	29	+def upload(stream, files, src_dir, file_exts, checkin=False, log=None):
	30	+ """Checkin / Upload files to the server."""
	31	+
	32	+ def is_adj(path):
	33	+ return False if len(os.path.dirname(path)) == 0 else True
	34	+
	35	+ log_msg(_("Sending request..."), log)
	36	+ if checkin:
	37	+ write_msg(stream, CheckinRequest(len(files)))
	38	+ else:
	39	+ write_msg(stream, UploadRequest(len(files)))
	40	+ done = []
	41	+
	42	+ failed = set()
	43	+ for file_path in files:
	44	+ log_msg(_("Uploading") + " " + file_path + "...", log)
	45	+
	46	+ if not is_adj(file_path):
	47	+ file_id = file_path
	48	+ contents = read_contents(src_dir, file_path, file_exts)
	49	+ else:
	50	+ file_id, choice = os.path.split(file_path)
	51	+ contents = read_contents(os.path.join(src_dir, file_id), choice, file_exts)
	52	+
	53	+ write_msg(stream, Message(file_id))
	54	+ write_msg(stream, BoolMessage(is_adj(file_path)))
	55	+ msg = read_msg(stream)
	56	+ if type(msg) != OkMessage:
	57	+ log_msg(_("Upload failed: server doesn't accept this file from you"), log)
	58	+ failed.add(file_path)
	59	+ continue
	60	+ send_contents(stream, contents)
	61	+
	62	+ write_msg(stream, ClientDone())
	63	+ accept_msg(stream, ServerDone)
	64	+
	65	+ if not checkin:
	66	+ log_msg(_("Done"), log)
	67	+ return
	68	+
	69	+ # When checkin command
	70	+ for file_path in files:
	71	+ if file_path in failed:
	72	+ continue
	73	+ if not is_adj(file_path):
	74	+ log_msg(_("Deleting") + " " + file_path + "...", log)
	75	+ for ext in file_exts:
	76	+ os.remove(os.path.join(src_dir, file_path + ext))
	77	+ else:
	78	+ file_id, ver = os.path.split(file_path)
	79	+ log_msg(_("Deleting") + " " + file_id + "...", log)
	80	+ shutil.rmtree(os.path.join(src_dir, file_id))
	81	+
	82	+ log_msg(_("Done"), log)
	83	+
	84	+def download(stream, n, dest_dir, tmp_dir, log=None):
	85	+ """Download from server n files."""
	86	+
	87	+ log_msg(_("Sending request..."), log)
	88	+ write_msg(stream, DownloadRequest(n))
	89	+ n = accept_msg(stream, NumMessage).get_number()
	90	+ if n == 0:
	91	+ log_msg(_("No files to download"), log)
	92	+
	93	+ done = []
	94	+ for i in range(n):
	95	+ file_id = accept_msg(stream, Message).get_contents()
	96	+ log_msg(_("Downloading") + " " + file_id + "...", log)
	97	+ contents = receive_contents(stream)
	98	+ save_contents(contents, tmp_dir, file_id)
	99	+ for ext in contents.keys():
	100	+ done.append(file_id + ext)
	101	+
	102	+ write_msg(stream, ClientDone())
	103	+ accept_msg(stream, ServerDone)
	104	+ for file_name in done:
	105	+ from_path = os.path.join(tmp_dir, file_name)
	106	+ to_path = os.path.join(dest_dir, file_name)
	107	+ shutil.move(from_path, to_path)
	108	+ log_msg(_("Done"), log)
	109	+
	110	+def download_prim(stream, n, dest_dir, tmp_dir, log=None):
	111	+ """Download from server n pairs of files for adjudication."""
	112	+
	113	+ log_msg(_("Sending request..."), log)
	114	+ write_msg(stream, DownloadPrimRequest(n))
	115	+
	116	+ msg = read_msg(stream)
	117	+ if type(msg) != OkMessage:
	118	+ log_msg(_("FAILED: you don't have adjudicator privileges"), log)
	119	+ return
	120	+
	121	+ n = accept_msg(stream, NumMessage).get_number()
	122	+ anno_per_file = accept_msg(stream, NumMessage).get_number()
	123	+ if n == 0:
	124	+ log_msg(_("No files to download"), log)
	125	+
	126	+ done = []
	127	+ for i in range(n):
	128	+ file_id = accept_msg(stream, Message).get_contents()
	129	+ log_msg(_("Downloading") + " " + file_id + "...", log)
	130	+
	131	+ file_dir = os.path.join(tmp_dir, file_id)
	132	+ os.mkdir(file_dir)
	133	+ done.append(file_id)
	134	+
	135	+ for j in range(anno_per_file):
	136	+ save_contents(receive_contents(stream), file_dir, chr(ord('A') + j))
	137	+
	138	+ if accept_msg(stream, BoolMessage).get_boolean() == True:
	139	+ if anno_per_file == 1:
	140	+ save_contents(receive_contents(stream), file_dir, "A")
	141	+ else:
	142	+ save_contents(receive_contents(stream), file_dir, "Super")
	143	+
	144	+ write_msg(stream, ClientDone())
	145	+ accept_msg(stream, ServerDone)
	146	+ for fild_id in done:
	147	+ from_path = os.path.join(tmp_dir, fild_id)
	148	+ to_path = os.path.join(dest_dir, fild_id)
	149	+ shutil.move(from_path, to_path)
	150	+ log_msg(_("Done"), log)
	151	+
	152	+def check_stats(stream, log=None):
	153	+ """Check annotatator stats."""
	154	+
	155	+ log_msg(_("Sending request..."), log)
	156	+ write_msg(stream, StatsRequest())
	157	+ n = accept_msg(stream, NumMessage).get_number()
	158	+ log_msg(_("Number of finished files: ") + str(n), log)
	159	+ write_msg(stream, ClientDone())
	160	+ accept_msg(stream, ServerDone)
	161	+ log_msg(_("Done"), log)
	162	+
	163	+def checkout(stream, dest_dir, tmp_dir, exts, log=None):
	164	+ log_msg(_("Sending request..."), log)
	165	+ write_msg(stream, CheckoutRequest())
	166	+
	167	+ done = []
	168	+ n = accept_msg(stream, NumMessage).get_number()
	169	+ for i in range(n):
	170	+ file_id = accept_msg(stream, Message).get_contents()
	171	+ log_msg(_("Downloading") + " " + file_id + "...", log)
	172	+
	173	+ contents = receive_contents(stream)
	174	+ save_contents(contents, tmp_dir, file_id)
	175	+ for ext in contents.keys():
	176	+ done.append(file_id + ext)
	177	+
	178	+ n = accept_msg(stream, NumMessage).get_number()
	179	+ anno_per_file = accept_msg(stream, NumMessage).get_number()
	180	+
	181	+ for i in range(n):
	182	+ file_id = accept_msg(stream, Message).get_contents()
	183	+ log_msg(_("Downloading") + " " + file_id + "...", log)
	184	+
	185	+ file_dir = os.path.join(tmp_dir, file_id)
	186	+ os.mkdir(file_dir)
	187	+
	188	+ for j in range(anno_per_file):
	189	+ save_contents(receive_contents(stream), file_dir, chr(j + ord('A')))
	190	+
	191	+ if accept_msg(stream, BoolMessage).get_boolean() == True:
	192	+ if anno_per_file == 1:
	193	+ save_contents(receive_contents(stream), file_dir, "A")
	194	+ else:
	195	+ save_contents(receive_contents(stream), file_dir, "Super")
	196	+
	197	+ done.append(file_id)
	198	+
	199	+ write_msg(stream, ClientDone())
	200	+ accept_msg(stream, ServerDone)
	201	+
	202	+ log_msg(_("Deleting local files..."), log)
	203	+ try:
	204	+ # normal texts
	205	+ for f in normal_files(dest_dir, exts):
	206	+ for ext in exts:
	207	+ os.remove(os.path.join(dest_dir, f + ext))
	208	+
	209	+ # super annotated texts
	210	+ dirs = set()
	211	+ for f in super_files(dest_dir, exts):
	212	+ dir = f.split(os.path.sep)[0]
	213	+ dirs.add(dir)
	214	+ for dir in dirs:
	215	+ shutil.rmtree(os.path.join(dest_dir, dir))
	216	+
	217	+ except Exception as ex:
	218	+ log_msg(_("Error occured:") + unicode(str(ex), errors='replace'), log)
	219	+
	220	+ log_msg(_("Saving dowloaded files..."), log)
	221	+ for path in done:
	222	+ from_path = os.path.join(tmp_dir, path)
	223	+ to_path = os.path.join(dest_dir, path)
	224	+ shutil.move(from_path, to_path)
	225	+
	226	+ log_msg(_("Done"), log)
	227	+
	228	+def return_files(stream, files, reason, src_dir, file_exts, log=None):
	229	+ """Return files to the server."""
	230	+
	231	+ def is_adj(path):
	232	+ return False if len(os.path.dirname(path)) == 0 else True
	233	+
	234	+ log_msg(_("Sending request..."), log)
	235	+ write_msg(stream, ReturnRequest(len(files)))
	236	+ write_msg(stream, Message(reason))
	237	+
	238	+ done = []
	239	+
	240	+ failed = set()
	241	+ for file_path in files:
	242	+ log_msg(_("Uploading") + " " + file_path + "...", log)
	243	+
	244	+ if not is_adj(file_path):
	245	+ file_id = file_path
	246	+ contents = read_contents(src_dir, file_path, file_exts)
	247	+ else:
	248	+ file_id, choice = os.path.split(file_path)
	249	+ contents = read_contents(os.path.join(src_dir, file_id), choice, file_exts)
	250	+
	251	+ write_msg(stream, Message(file_id))
	252	+ write_msg(stream, BoolMessage(is_adj(file_path)))
	253	+ msg = read_msg(stream)
	254	+ if type(msg) != OkMessage:
	255	+ log_msg(_("Upload failed: server doesn't accept this file from you"), log)
	256	+ failed.add(file_path)
	257	+ continue
	258	+ send_contents(stream, contents)
	259	+
	260	+ write_msg(stream, ClientDone())
	261	+ accept_msg(stream, ServerDone)
	262	+
	263	+ for file_path in files:
	264	+ if file_path in failed:
	265	+ continue
	266	+ if not is_adj(file_path):
	267	+ log_msg(_("Deleting") + " " + file_path + "...", log)
	268	+ for ext in file_exts:
	269	+ os.remove(os.path.join(src_dir, file_path + ext))
	270	+ else:
	271	+ file_id, ver = os.path.split(file_path)
	272	+ log_msg(_("Deleting") + " " + file_id + "...", log)
	273	+ shutil.rmtree(os.path.join(src_dir, file_id))
	274	+
	275	+ log_msg(_("Done"), log)
	276	+
	277	+def save_contents(contents, dest_dir, file_id):
	278	+ for ext, data in contents.iteritems():
	279	+ dest_path = os.path.join(dest_dir, file_id + ext)
	280	+ with open(dest_path, "w") as dest:
	281	+ dest.write(data)
	282	+
	283	+def read_contents(src_dir, file_id, file_exts):
	284	+ result = {}
	285	+ regex = re.compile("%s(.*)$" % file_id)
	286	+ for entry in os.listdir(src_dir):
	287	+ match = regex.match(entry)
	288	+ if not match:
	289	+ continue
	290	+ with open(os.path.join(src_dir, entry)) as src:
	291	+ ext = match.group(1)
	292	+ if ext in file_exts:
	293	+ result[ext] = src.read()
	294	+ else:
	295	+ print "Skipping file " + entry
	296	+ return result
	297	+
	298	+def run_auth(stream, login, passwd, log=None):
	299	+ log_msg(_("Authentication..."), log)
	300	+ write_msg(stream, Message(login))
	301	+ write_msg(stream, Message(passwd))
	302	+ msg = read_msg(stream)
	303	+ if type(msg) != OkMessage:
	304	+ log_msg(_("FAILED: incorrect login or password"), log)
	305	+ return False
	306	+ return True
	307	+
	308	+def run_upload(login, passwd, host, port, cert, files, src_dir, file_exts, checkin=False, log=None):
	309	+ result = True
	310	+ try:
	311	+ stream = init_stream(host, port, cert, log=log)
	312	+ try:
	313	+ if run_auth(stream, login, passwd, log=log):
	314	+ upload(stream, files, src_dir, file_exts, checkin=checkin, log=log)
	315	+ finally:
	316	+ stream.close()
	317	+ except:
	318	+ log_msg(_("FAILED: unable to connect to server"), log)
	319	+ print traceback.format_exc()
	320	+ result = False
	321	+ if log:
	322	+ log.put(None)
	323	+ return result
	324	+
	325	+def run_download(login, passwd, host, port, cert, n, dest_dir, log=None):
	326	+ result = True
	327	+ try:
	328	+ stream = init_stream(host, port, cert, log=log)
	329	+ try:
	330	+ tmp_dir = tempfile.mkdtemp()
	331	+ try:
	332	+ if run_auth(stream, login, passwd, log=log):
	333	+ download(stream, n, dest_dir, tmp_dir, log=log)
	334	+ finally:
	335	+ shutil.rmtree(tmp_dir)
	336	+ finally:
	337	+ stream.close()
	338	+ except:
	339	+ log_msg(_("FAILED: unable to connect to server"), log)
	340	+ print traceback.format_exc()
	341	+ result = False
	342	+ if log:
	343	+ log.put(None)
	344	+ return result
	345	+
	346	+def run_check_stats(login, passwd, host, port, cert, log=None):
	347	+ result = True
	348	+ try:
	349	+ stream = init_stream(host, port, cert, log=log)
	350	+ try:
	351	+ if run_auth(stream, login, passwd, log=log):
	352	+ check_stats(stream, log=log)
	353	+ finally:
	354	+ stream.close()
	355	+ except:
	356	+ log_msg(_("FAILED: unable to connect to server"), log)
	357	+ print traceback.format_exc()
	358	+ result = False
	359	+ if log:
	360	+ log.put(None)
	361	+ return result
	362	+
	363	+def run_download_prim(login, passwd, host, port, cert, n, dest_dir, log=None):
	364	+ result = True
	365	+ try:
	366	+ stream = init_stream(host, port, cert, log=log)
	367	+ try:
	368	+ tmp_dir = tempfile.mkdtemp()
	369	+ try:
	370	+ if run_auth(stream, login, passwd, log=log):
	371	+ download_prim(stream, n, dest_dir, tmp_dir, log=log)
	372	+ finally:
	373	+ shutil.rmtree(tmp_dir)
	374	+ finally:
	375	+ stream.close()
	376	+ except:
	377	+ log_msg(_("FAILED: unable to connect to server"), log)
	378	+ print traceback.format_exc()
	379	+ result = False
	380	+ if log:
	381	+ log.put(None)
	382	+ return result
	383	+
	384	+def run_checkout(login, passwd, host, port, cert, dest_dir, exts, log=None):
	385	+ result = True
	386	+ try:
	387	+ stream = init_stream(host, port, cert, log=log)
	388	+ try:
	389	+ tmp_dir = tempfile.mkdtemp()
	390	+ try:
	391	+ if run_auth(stream, login, passwd, log=log):
	392	+ checkout(stream, dest_dir, tmp_dir, exts, log=log)
	393	+ finally:
	394	+ shutil.rmtree(tmp_dir)
	395	+ finally:
	396	+ stream.close()
	397	+ except:
	398	+ log_msg(_("FAILED: unable to connect to server"), log)
	399	+ print traceback.format_exc()
	400	+ result = False
	401	+ if log:
	402	+ log.put(None)
	403	+ return result
	404	+
	405	+def run_return(login, passwd, host, port, cert, files, reason, src_dir, file_exts, log=None):
	406	+ result = True
	407	+ try:
	408	+ stream = init_stream(host, port, cert, log=log)
	409	+ try:
	410	+ if run_auth(stream, login, passwd, log=log):
	411	+ return_files(stream, files, reason, src_dir, file_exts, log)
	412	+ finally:
	413	+ stream.close()
	414	+ except:
	415	+ log_msg(_("FAILED: unable to connect to server"), log)
	416	+ print traceback.format_exc()
	417	+ result = False
	418	+ if log:
	419	+ log.put(None)
	420	+ return result
	421	+
	422	+def log_msg(msg, log, new_line=True):
	423	+ if new_line:
	424	+ msg = msg + "\n"
	425	+ sys.stdout.write(msg.encode('utf-8'))
	426	+ if log:
	427	+ log.put(msg)
	428	+
	429	+if __name__ == "__main__":
	430	+
	431	+ optparser = OptionParser(usage="""usage: %prog [options] CMD CMD-ARGS
	432	+
	433	+ Command line client for files distribution system.""")
	434	+
	435	+ optparser.add_option("--login", dest="login")
	436	+ optparser.add_option("--passwd", dest="passwd")
	437	+ optparser.add_option("--home", metavar="DIR", dest="home",
	438	+ help="Directory with users files.")
	439	+
	440	+ (options, args) = optparser.parse_args()
	441	+
	442	+ if options.login == None:
	443	+ optparser.print_help()
	444	+ print "\n--login option is mandatory"
	445	+ sys.exit(0)
	446	+ if options.passwd == None:
	447	+ optparser.print_help()
	448	+ print "\n--passwd option is mandatory"
	449	+ sys.exit(0)
	450	+ if options.home == None:
	451	+ optparser.print_help()
	452	+ print "\n--home option is mandatory"
	453	+ sys.exit(0)
	454	+
	455	+ commands = ["upload", "download", "download2"]
	456	+ if len(args) < 1 or args[0] not in commands:
	457	+ optparser.print_help()
	458	+ print "\nCMD should be one of the following:"
	459	+ print commands
	460	+ sys.exit(0)
	461	+
	462	+ cmd = args[0]
	463	+ if cmd == "upload":
	464	+ run_upload(options.login, options.passwd, args[1:], options.home)
	465	+ elif cmd == "download":
	466	+ run_download(options.login, options.passwd,
	467	+ int(args[1]), options.home)
	468	+ elif cmd == "download2":
	469	+ run_download_prim(options.login, options.passwd,
	470	+ int(args[1]), options.home)
...	...

gui/const.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/const.py
	1	+# -- coding: utf-8 --
	2	+import os
	3	+
	4	+from dfs.config import Config
	5	+
	6	+# Global configuration file
	7	+global_config_name = "data/global.cfg"
	8	+_cfg = Config(global_config_name)
	9	+
	10	+# File with list of directories
	11	+dir_cache_name = _cfg["general.dir_cache"]
	12	+
	13	+# File with directory configuration
	14	+config_name = _cfg["general.config"]
	15	+# File with file names marked as ready
	16	+ready_name = _cfg["general.ready"]
	17	+# File with files footprints
	18	+footprints_name = _cfg["general.footprints"]
	19	+# Default file extensions
	20	+default_exts = _cfg["general.default_exts"].split()
	21	+# Language (pl or en)
	22	+lang = _cfg["general.lang"]
	23	+# option to hide dir menu if precofigured
	24	+hide_dir_menu = _cfg["general.hide_dir_menu"]
	25	+
	26	+program = _cfg["general.program"]
	27	+
	28	+extension = _cfg["general.extension"]
	29	+
	30	+conn_host = _cfg["connection.host"]
	31	+conn_port = int(_cfg["connection.port"])
	32	+conn_cert = _cfg["connection.cert"]
	33	+
	34	+# Path to directory-specific configuration files
	35	+def path(p, name):
	36	+ return os.path.join(p, name)
...	...

gui/data/.dir.lst 0 → 100644

View file @44cbbee

	1	+++ a/gui/data/.dir.lst
	1	+/home/me2/workspace/dist-system/data/foldery_anotatorow/ann1
	2	+/home/me2/workspace/dist-system/data/foldery_anotatorow
	3	+/home/me2/workspace/core/dist-system/data/annotators_folders/ann2
	4	+/home/me2/workspace/core/dist-system/data/annotators_folders/ann1
	5	+/home/me2/workspace/core/dist-system/data/annotators_folders/adj1
...	...

gui/data/cert.pem 0 → 100644

View file @44cbbee

	1	+++ a/gui/data/cert.pem
	1	+-----BEGIN CERTIFICATE-----
	2	+MIICkDCCAfmgAwIBAgIJAJPcESSEV86nMA0GCSqGSIb3DQEBBQUAMGExCzAJBgNV
	3	+BAYTAlBMMQ0wCwYDVQQIDARUZXN0MQ0wCwYDVQQHDARDaXR5MRAwDgYDVQQKDAdD
	4	+b21wYW55MSIwIAYJKoZIhvcNAQkBFhNleGFtcGxlQGV4YW1wbGUuY29tMB4XDTEz
	5	+MDEyMzEyMjIwN1oXDTE0MDEyMzEyMjIwN1owYTELMAkGA1UEBhMCUEwxDTALBgNV
	6	+BAgMBFRlc3QxDTALBgNVBAcMBENpdHkxEDAOBgNVBAoMB0NvbXBhbnkxIjAgBgkq
	7	+hkiG9w0BCQEWE2V4YW1wbGVAZXhhbXBsZS5jb20wgZ8wDQYJKoZIhvcNAQEBBQAD
	8	+gY0AMIGJAoGBAKUG9QHjUiCSjcvFQl1OqUgZ8c4Vzm7m7Ua7D33YDnGo3QGuhPML
	9	+0JB+JPBG++MrPdQqMV8eeXe1dFlrm6SHnrEZIhwZg8iDnPhFKNjHiAkx0PCQdI93
	10	+rAe4EcszkgbQCXXLGjQJNDikwGZ6ciylJ7C2XewWTlWjv7/LqMwhQotLAgMBAAGj
	11	+UDBOMB0GA1UdDgQWBBRLRvOFIo9LIeiT2ohQalEzy687ijAfBgNVHSMEGDAWgBRL
	12	+RvOFIo9LIeiT2ohQalEzy687ijAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUA
	13	+A4GBAD1odtgH1kZ4a0TEbwjK2PaHTYyofkkgeXsGGnG3eMkwKQ7PHJarBl7GtxP5
	14	+Ls91k/KvBuNqCM/lPmM94USDSXrKGe0Zzc9WZbDG9SXO23gGOwPl+Tg9v1ocRHd2
	15	+dYYIgZM8/tLcsHyNNMd7c4BOMD4rzkjgbThFFMl/2NSkvrO7
	16	+-----END CERTIFICATE-----
...	...

gui/data/global.cfg 0 → 100644

View file @44cbbee

	1	+++ a/gui/data/global.cfg
	1	+[general]
	2	+
	3	+config = .dir.cfg
	4	+ready = .ready.lst
	5	+footprints = .footprints.lst
	6	+default_exts = .mmax _mentions.xml _words.xml
	7	+lang = pl
	8	+hide_dir_menu = no
	9	+program = mmax
	10	+extension = .mmax
	11	+
	12	+{paths}
	13	+dir_cache = .dir.lst
	14	+
	15	+[connection]
	16	+
	17	+host = localhost
	18	+port = 2225
	19	+
	20	+{paths}
	21	+cert = cert.pem
...	...

gui/dfs.wxg 0 → 100644

View file @44cbbee

	1	+++ a/gui/dfs.wxg
	1	+<?xml version="1.0"?>
	2	+<!-- generated by wxGlade 0.6.3 on Fri Sep 30 18:28:38 2011 -->
	3	+
	4	+<application path="/home/kuba/Pulpit/praca/nkjp/dist-system/gui" name="manager" class="FileManager" option="1" language="python" top_window="main_frame" encoding="UTF-8" use_gettext="0" overwrite="0" use_new_namespace="1" for_version="2.8" is_template="0">
	5	+ <object class="MainFrame" name="main_frame" base="EditFrame">
	6	+ <style>wxDEFAULT_FRAME_STYLE</style>
	7	+ <title>--</title>
	8	+ <menubar>1</menubar>
	9	+ <centered>1</centered>
	10	+ <focused>1</focused>
	11	+ <size>663, 556</size>
	12	+ <object class="wxMenuBar" name="frame_1_menubar" base="EditMenuBar">
	13	+ <menus>
	14	+ <menu name="dir_menu" label="Directory">
	15	+ <item>
	16	+ <label>Open</label>
	17	+ <id>5</id>
	18	+ <name>dir_menu_open</name>
	19	+ <help_str>Open new directory</help_str>
	20	+ <handler>open_directory</handler>
	21	+ </item>
	22	+ <item>
	23	+ <label>Reconfigure</label>
	24	+ <id>6</id>
	25	+ <help_str>Reconfigure directory</help_str>
	26	+ <handler>reconf_directory</handler>
	27	+ </item>
	28	+ <item>
	29	+ <label>---</label>
	30	+ <id>---</id>
	31	+ <name>---</name>
	32	+ </item>
	33	+ </menu>
	34	+ <menu name="" label="Files">
	35	+ <item>
	36	+ <label>Upload Selected</label>
	37	+ <id>1</id>
	38	+ <help_str>Upload annotated or adjudicated files</help_str>
	39	+ <handler>upload</handler>
	40	+ </item>
	41	+ <item>
	42	+ <label>Checkin Selected</label>
	43	+ <id>8</id>
	44	+ <help_str>Upload annotated or adjudicated; files will be deleted from local disk and you will not be able to modify them again</help_str>
	45	+ <handler>checkin</handler>
	46	+ </item>
	47	+ <item>
	48	+ <label>Download (annotation)</label>
	49	+ <id>2</id>
	50	+ <help_str>Download files for annotation</help_str>
	51	+ <handler>download</handler>
	52	+ </item>
	53	+ <item>
	54	+ <label>Download (adjudication)</label>
	55	+ <id>3</id>
	56	+ <help_str>Download files for adjudication</help_str>
	57	+ <handler>download_prim</handler>
	58	+ </item>
	59	+ <item>
	60	+ <label>Checkout</label>
	61	+ <id>7</id>
	62	+ <help_str>Checkout files from server to your local, empty directory</help_str>
	63	+ <handler>checkout</handler>
	64	+ </item>
	65	+ </menu>
	66	+ <menu name="" label="Select">
	67	+ <item>
	68	+ <label>Ready</label>
	69	+ <id>4</id>
	70	+ <help_str>Select items which are marked for upload</help_str>
	71	+ <handler>select_ready</handler>
	72	+ </item>
	73	+ <item>
	74	+ <label>Modified</label>
	75	+ <id>9</id>
	76	+ <help_str>Select items which are modified</help_str>
	77	+ <handler>select_modified</handler>
	78	+ </item>
	79	+ </menu>
	80	+ <menu name="" label="Help">
	81	+ </menu>
	82	+ </menus>
	83	+ </object>
	84	+ <object class="wxBoxSizer" name="sizer_6" base="EditBoxSizer">
	85	+ <orient>wxHORIZONTAL</orient>
	86	+ <object class="sizeritem">
	87	+ <flag>wxEXPAND</flag>
	88	+ <border>0</border>
	89	+ <option>1</option>
	90	+ <object class="wxSplitterWindow" name="window_1" base="EditSplitterWindow">
	91	+ <style>wxSP_3D\|wxSP_BORDER</style>
	92	+ <orientation>wxSPLIT_HORIZONTAL</orientation>
	93	+ <sash_pos>-100</sash_pos>
	94	+ <window_2>window_1_pane_2</window_2>
	95	+ <window_1>window_1_pane_1</window_1>
	96	+ <object class="wxPanel" name="window_1_pane_1" base="EditPanel">
	97	+ <style>wxTAB_TRAVERSAL</style>
	98	+ <object class="wxBoxSizer" name="sizer_1" base="EditBoxSizer">
	99	+ <orient>wxVERTICAL</orient>
	100	+ <object class="sizeritem">
	101	+ <flag>wxEXPAND</flag>
	102	+ <border>0</border>
	103	+ <option>1</option>
	104	+ <object class="wxNotebook" name="notebook_1" base="EditNotebook">
	105	+ <style>0</style>
	106	+ <tabs>
	107	+ <tab window="notebook_1_pane_1">annotation</tab>
	108	+ <tab window="notebook_1_pane_2">adjudication</tab>
	109	+ </tabs>
	110	+ <object class="wxPanel" name="notebook_1_pane_1" base="EditPanel">
	111	+ <style>wxTAB_TRAVERSAL</style>
	112	+ <object class="wxBoxSizer" name="sizer_2" base="EditBoxSizer">
	113	+ <orient>wxVERTICAL</orient>
	114	+ <object class="sizeritem">
	115	+ <flag>wxEXPAND</flag>
	116	+ <border>0</border>
	117	+ <option>1</option>
	118	+ <object class="wxListCtrl" name="files_list" base="EditListCtrl">
	119	+ <style>wxLC_REPORT\|wxSIMPLE_BORDER</style>
	120	+ <events>
	121	+ <handler event="EVT_LIST_ITEM_ACTIVATED">activate_item</handler>
	122	+ </events>
	123	+ </object>
	124	+ </object>
	125	+ </object>
	126	+ </object>
	127	+ <object class="wxPanel" name="notebook_1_pane_2" base="EditPanel">
	128	+ <style>wxTAB_TRAVERSAL</style>
	129	+ <object class="wxBoxSizer" name="sizer_3" base="EditBoxSizer">
	130	+ <orient>wxVERTICAL</orient>
	131	+ <object class="sizeritem">
	132	+ <flag>wxEXPAND</flag>
	133	+ <border>0</border>
	134	+ <option>1</option>
	135	+ <object class="wxListCtrl" name="super_files_list" base="EditListCtrl">
	136	+ <style>wxLC_REPORT\|wxSIMPLE_BORDER</style>
	137	+ <events>
	138	+ <handler event="EVT_LIST_ITEM_ACTIVATED">activate_super_item</handler>
	139	+ </events>
	140	+ </object>
	141	+ </object>
	142	+ </object>
	143	+ </object>
	144	+ </object>
	145	+ </object>
	146	+ </object>
	147	+ </object>
	148	+ <object class="wxPanel" name="window_1_pane_2" base="EditPanel">
	149	+ <style>wxTAB_TRAVERSAL</style>
	150	+ <object class="wxBoxSizer" name="sizer_7" base="EditBoxSizer">
	151	+ <orient>wxVERTICAL</orient>
	152	+ <object class="sizeritem">
	153	+ <flag>wxEXPAND</flag>
	154	+ <border>0</border>
	155	+ <option>1</option>
	156	+ <object class="wxTextCtrl" name="log_ctrl" base="EditTextCtrl">
	157	+ <style>wxTE_MULTILINE\|wxTE_READONLY</style>
	158	+ </object>
	159	+ </object>
	160	+ </object>
	161	+ </object>
	162	+ </object>
	163	+ </object>
	164	+ </object>
	165	+ </object>
	166	+ <object class="LoginDialog" name="login_dialog" base="EditDialog">
	167	+ <style>wxDEFAULT_DIALOG_STYLE\|wxRESIZE_BORDER\|wxTHICK_FRAME</style>
	168	+ <title>login</title>
	169	+ <object class="wxBoxSizer" name="sizer_4" base="EditBoxSizer">
	170	+ <orient>wxVERTICAL</orient>
	171	+ <object class="sizeritem">
	172	+ <border>0</border>
	173	+ <option>0</option>
	174	+ <object class="wxGridSizer" name="grid_sizer_1" base="EditGridSizer">
	175	+ <hgap>0</hgap>
	176	+ <rows>2</rows>
	177	+ <cols>2</cols>
	178	+ <vgap>0</vgap>
	179	+ <object class="sizeritem">
	180	+ <flag>wxALIGN_CENTER_VERTICAL</flag>
	181	+ <border>0</border>
	182	+ <option>0</option>
	183	+ <object class="wxStaticText" name="label_1" base="EditStaticText">
	184	+ <attribute>1</attribute>
	185	+ <label>login: </label>
	186	+ </object>
	187	+ </object>
	188	+ <object class="sizeritem">
	189	+ <border>0</border>
	190	+ <option>0</option>
	191	+ <object class="wxTextCtrl" name="login_ctrl" base="EditTextCtrl">
	192	+ </object>
	193	+ </object>
	194	+ <object class="sizeritem">
	195	+ <flag>wxALIGN_CENTER_VERTICAL</flag>
	196	+ <border>0</border>
	197	+ <option>0</option>
	198	+ <object class="wxStaticText" name="label_3" base="EditStaticText">
	199	+ <attribute>1</attribute>
	200	+ <label>password: </label>
	201	+ </object>
	202	+ </object>
	203	+ <object class="sizeritem">
	204	+ <border>0</border>
	205	+ <option>0</option>
	206	+ <object class="wxTextCtrl" name="passwd_ctrl" base="EditTextCtrl">
	207	+ <style>wxTE_PASSWORD</style>
	208	+ </object>
	209	+ </object>
	210	+ </object>
	211	+ </object>
	212	+ <object class="sizeritem">
	213	+ <border>0</border>
	214	+ <option>0</option>
	215	+ <object class="wxCheckBox" name="save_passwd" base="EditCheckBox">
	216	+ <label>save password</label>
	217	+ </object>
	218	+ </object>
	219	+ <object class="sizeritem">
	220	+ <flag>wxALIGN_CENTER_HORIZONTAL</flag>
	221	+ <border>0</border>
	222	+ <option>0</option>
	223	+ <object class="wxButton" name="button_1" base="EditButton">
	224	+ <label>OK</label>
	225	+ <id>wx.ID_OK</id>
	226	+ <events>
	227	+ <handler event="EVT_BUTTON">on_ok</handler>
	228	+ </events>
	229	+ <size>85, 32</size>
	230	+ </object>
	231	+ </object>
	232	+ </object>
	233	+ </object>
	234	+ <object class="DownloadNumberDialog" name="download_number" base="EditDialog">
	235	+ <style>wxDEFAULT_DIALOG_STYLE</style>
	236	+ <title>checkout</title>
	237	+ <centered>1</centered>
	238	+ <object class="wxGridSizer" name="grid_sizer_2" base="EditGridSizer">
	239	+ <hgap>0</hgap>
	240	+ <rows>2</rows>
	241	+ <cols>2</cols>
	242	+ <vgap>0</vgap>
	243	+ <object class="sizeritem">
	244	+ <flag>wxALIGN_CENTER_HORIZONTAL\|wxALIGN_CENTER_VERTICAL</flag>
	245	+ <border>0</border>
	246	+ <option>0</option>
	247	+ <object class="wxStaticText" name="label_4" base="EditStaticText">
	248	+ <attribute>1</attribute>
	249	+ <tooltip>Number of files to download</tooltip>
	250	+ <label>download:</label>
	251	+ </object>
	252	+ </object>
	253	+ <object class="sizeritem">
	254	+ <border>0</border>
	255	+ <option>0</option>
	256	+ <object class="wxSpinCtrl" name="checkout_number" base="EditSpinCtrl">
	257	+ <range>1, 300</range>
	258	+ <value>1</value>
	259	+ </object>
	260	+ </object>
	261	+ <object class="sizeritem">
	262	+ <flag>wxALIGN_CENTER_HORIZONTAL\|wxALIGN_CENTER_VERTICAL</flag>
	263	+ <border>0</border>
	264	+ <option>0</option>
	265	+ <object class="wxButton" name="button_4" base="EditButton">
	266	+ <label>OK</label>
	267	+ <id>wx.ID_YES</id>
	268	+ <events>
	269	+ <handler event="EVT_BUTTON">on_ok</handler>
	270	+ </events>
	271	+ </object>
	272	+ </object>
	273	+ <object class="sizeritem">
	274	+ <flag>wxALIGN_CENTER_HORIZONTAL\|wxALIGN_CENTER_VERTICAL</flag>
	275	+ <border>0</border>
	276	+ <option>0</option>
	277	+ <object class="wxButton" name="button_5" base="EditButton">
	278	+ <label>Cancel</label>
	279	+ <id>wx.ID_NO</id>
	280	+ <events>
	281	+ <handler event="EVT_BUTTON">on_cancel</handler>
	282	+ </events>
	283	+ </object>
	284	+ </object>
	285	+ </object>
	286	+ </object>
	287	+ <object class="ConfigDialog" name="config_dialog" base="EditDialog">
	288	+ <style>wxDEFAULT_DIALOG_STYLE</style>
	289	+ <title>Directory configuration</title>
	290	+ <object class="wxBoxSizer" name="sizer_5" base="EditBoxSizer">
	291	+ <orient>wxVERTICAL</orient>
	292	+ <object class="sizeritem">
	293	+ <border>0</border>
	294	+ <option>0</option>
	295	+ <object class="wxGridSizer" name="grid_sizer_4" base="EditGridSizer">
	296	+ <hgap>0</hgap>
	297	+ <rows>3</rows>
	298	+ <cols>2</cols>
	299	+ <vgap>0</vgap>
	300	+ <object class="sizeritem">
	301	+ <flag>wxALIGN_CENTER_VERTICAL</flag>
	302	+ <border>0</border>
	303	+ <option>0</option>
	304	+ <object class="wxStaticText" name="label_2" base="EditStaticText">
	305	+ <attribute>1</attribute>
	306	+ <tooltip>Login supplied by system administrator</tooltip>
	307	+ <label>login:</label>
	308	+ </object>
	309	+ </object>
	310	+ <object class="sizeritem">
	311	+ <flag>wxALIGN_CENTER_VERTICAL</flag>
	312	+ <border>0</border>
	313	+ <option>0</option>
	314	+ <object class="wxTextCtrl" name="login_ctrl" base="EditTextCtrl">
	315	+ </object>
	316	+ </object>
	317	+ <object class="sizeritem">
	318	+ <flag>wxALIGN_CENTER_VERTICAL</flag>
	319	+ <border>0</border>
	320	+ <option>0</option>
	321	+ <object class="wxStaticText" name="label_6" base="EditStaticText">
	322	+ <attribute>1</attribute>
	323	+ <tooltip>Password supplied by system administrator</tooltip>
	324	+ <label>password:</label>
	325	+ </object>
	326	+ </object>
	327	+ <object class="sizeritem">
	328	+ <flag>wxALIGN_CENTER_VERTICAL</flag>
	329	+ <border>0</border>
	330	+ <option>0</option>
	331	+ <object class="wxTextCtrl" name="passwd_ctrl" base="EditTextCtrl">
	332	+ <style>wxTE_PASSWORD</style>
	333	+ </object>
	334	+ </object>
	335	+ <object class="sizeritem">
	336	+ <flag>wxALIGN_CENTER_VERTICAL</flag>
	337	+ <border>0</border>
	338	+ <option>0</option>
	339	+ <object class="wxStaticText" name="label_5" base="EditStaticText">
	340	+ <attribute>1</attribute>
	341	+ <tooltip>List of space separated file extensions</tooltip>
	342	+ <label>extensions:</label>
	343	+ </object>
	344	+ </object>
	345	+ <object class="sizeritem">
	346	+ <flag>wxALIGN_CENTER_VERTICAL</flag>
	347	+ <border>0</border>
	348	+ <option>0</option>
	349	+ <object class="wxTextCtrl" name="exts_ctrl" base="EditTextCtrl">
	350	+ </object>
	351	+ </object>
	352	+ </object>
	353	+ </object>
	354	+ <object class="sizeritem">
	355	+ <flag>wxALIGN_CENTER_HORIZONTAL</flag>
	356	+ <border>0</border>
	357	+ <option>0</option>
	358	+ <object class="wxButton" name="button_2" base="EditButton">
	359	+ <label>OK</label>
	360	+ <id>wx.ID_OK</id>
	361	+ <focused>1</focused>
	362	+ <size>85, 29</size>
	363	+ </object>
	364	+ </object>
	365	+ </object>
	366	+ </object>
	367	+</application>
...	...

gui/dfs/__init__.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/dfs/__init__.py
...	...

gui/dfs/config.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/dfs/config.py
	1	+# -- coding: utf-8 --
	2	+import re
	3	+import os
	4	+
	5	+SECTION = re.compile("^\[(.*)\]$")
	6	+SUBSECTION = re.compile("^\{(.*)\}$")
	7	+ENTRY = re.compile("^([^=\s]+)\s=\s([^=]+)\s*$")
	8	+
	9	+def cut_comment(s):
	10	+ return s.split("#")[0]
	11	+
	12	+class Config:
	13	+
	14	+ def __init__(self, path):
	15	+ section = None
	16	+ subsection = None
	17	+ self.d = {}
	18	+ with open(path) as f:
	19	+ for line in f:
	20	+ line = cut_comment(line).strip()
	21	+
	22	+ m = SECTION.search(line)
	23	+ if m is not None:
	24	+ section = m.group(1)
	25	+ subsection = None
	26	+ continue
	27	+
	28	+ m = SUBSECTION.search(line)
	29	+ if m is not None:
	30	+ subsection = m.group(1)
	31	+ continue
	32	+
	33	+ m = ENTRY.search(line)
	34	+ if m is not None:
	35	+ key = m.group(1)
	36	+ val = m.group(2)
	37	+ if subsection == "paths":
	38	+ # print path, val, os.path.join(path, val)
	39	+ val = self.relative_path(val, path)
	40	+ if section is not None:
	41	+ self.d[section + "." + key] = val
	42	+ else:
	43	+ self.d[key] = val
	44	+
	45	+ def relative_path(self, path, ref):
	46	+ return os.path.join(os.path.dirname(ref), path)
	47	+
	48	+ def __getitem__(self, key):
	49	+ return self.d[key]
...	...

gui/dfs/database.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/dfs/database.py
	1	+# -- coding: utf-8 --
	2	+import time
	3	+from datetime import date, datetime
	4	+from collections import defaultdict
	5	+
	6	+from xml.etree.ElementTree import Element, ElementTree, tostring
	7	+import xml.etree.ElementTree as ET
	8	+
	9	+class Database:
	10	+
	11	+ def __init__(self, db, anno_per_file, from_file=True):
	12	+ if from_file:
	13	+ self.db_name = db
	14	+ self.tree = ET.parse(db)
	15	+ self.root = self.tree.getroot()
	16	+ else:
	17	+ self.db_name = None
	18	+ self.tree = None
	19	+ self.root = ET.fromstring(db)
	20	+
	21	+ self.anno_per_file = anno_per_file
	22	+ self.make_index()
	23	+
	24	+ def owned(self, ann_elem):
	25	+ return not self.finished(ann_elem) and not self.returned(ann_elem)
	26	+
	27	+ def finished(self, ann_elem):
	28	+ return ann_elem.find("checkinDate") is not None
	29	+
	30	+ def returned(self, ann_elem):
	31	+ return ann_elem.find("return") is not None
	32	+
	33	+ def get_reason(self, ann_elem):
	34	+ reas = ann_elem.find("return").find("reason").text
	35	+ if reas is None:
	36	+ reas = ""
	37	+ return reas
	38	+
	39	+ def rejected(self, file_elem):
	40	+ return file_elem.find("rejected") is not None
	41	+
	42	+ def fixed(self, ann_elem):
	43	+ ret = ann_elem.find("return")
	44	+ if ret is not None and ret.find("fixed") is not None:
	45	+ return True
	46	+ return False
	47	+
	48	+ def ann_elem_for_user(self, file_elem, user):
	49	+ idx = 0
	50	+ for ann_elem in file_elem.findall("ann"):
	51	+ if ann_elem.find("annName").text == user:
	52	+ return ann_elem, idx
	53	+ idx = idx + 1
	54	+ return None, None
	55	+
	56	+ def sann_elem_for_user(self, file_elem, user):
	57	+ for ann_elem in file_elem.findall("s_ann"):
	58	+ if ann_elem.find("annName").text == user:
	59	+ return ann_elem
	60	+ return None
	61	+
	62	+ def fixed_for_user(self, file_elem, user, is_adj):
	63	+ if is_adj:
	64	+ ann = self.sann_elem_for_user(file_elem, user)
	65	+ else:
	66	+ ann, idx = self.ann_elem_for_user(file_elem, user)
	67	+ if ann is not None:
	68	+ return self.fixed(ann)
	69	+ return False
	70	+
	71	+ def make_index(self):
	72	+ self.file_index = {}
	73	+ self.ann_owned_index = defaultdict(list)
	74	+ self.adj_owned_index = defaultdict(list)
	75	+
	76	+ for file_elem in self.root:
	77	+ name_elem = file_elem.find("name")
	78	+ if name_elem == None:
	79	+ raise Exception("No name assigned to a file element !")
	80	+ self.file_index[name_elem.text] = file_elem
	81	+
	82	+ if self.rejected(file_elem):
	83	+ continue
	84	+
	85	+ for ann_elem in file_elem.findall("ann"):
	86	+ if self.owned(ann_elem):
	87	+ ann_name = ann_elem.find("annName").text
	88	+ self.ann_owned_index[ann_name].append(file_elem)
	89	+
	90	+ for ann_elem in file_elem.findall("s_ann"):
	91	+ if self.owned(ann_elem):
	92	+ ann_name = ann_elem.find("annName").text
	93	+ self.adj_owned_index[ann_name].append(file_elem)
	94	+
	95	+ def fix(self, ann_elem):
	96	+ if not self.returned(ann_elem):
	97	+ raise Exception("File not returned in database!")
	98	+ if self.fixed(ann_elem):
	99	+ raise Exception("File already fixed in database!")
	100	+
	101	+ ret = ann_elem.find("return")
	102	+ fixed = ET.SubElement(ret, "fixed")
	103	+
	104	+ def add(self, file_name):
	105	+ if self.file_index.has_key(file_name):
	106	+ raise Exception("File name already in database !")
	107	+ file_elem = ET.SubElement(self.root, "file")
	108	+ name_elem = ET.SubElement(file_elem, "name")
	109	+ name_elem.text = file_name
	110	+ date_elem = ET.SubElement(file_elem, "addDate")
	111	+ date_elem.text = time.strftime("%c")
	112	+ self.file_index[file_name] = file_elem
	113	+
	114	+ def remove(self, file_name):
	115	+ if not self.file_index.has_key(file_name):
	116	+ raise Exception("File name not in database !")
	117	+ file_elem = self.file_index[file_name]
	118	+ self.root.remove(file_elem)
	119	+ self.make_index()
	120	+
	121	+ def reject(self, file_name, reason):
	122	+ if not self.file_index.has_key(file_name):
	123	+ raise Exception("File name not in database !")
	124	+ file_elem = self.file_index[file_name]
	125	+ if self.rejected(file_elem):
	126	+ raise Exception("File already rejected !")
	127	+ rej_elem = ET.SubElement(file_elem, "rejected")
	128	+ rej_elem.text = reason
	129	+ self.make_index()
	130	+
	131	+ # For prettyprint, used in 'save' method.
	132	+ def indent(self, elem, level=0):
	133	+ i = "\n" + level * " "
	134	+ if len(elem):
	135	+ if not elem.text or not elem.text.strip():
	136	+ elem.text = i + " "
	137	+ if not elem.tail or not elem.tail.strip():
	138	+ elem.tail = i
	139	+ for elem in elem:
	140	+ self.indent(elem, level + 1)
	141	+ if not elem.tail or not elem.tail.strip():
	142	+ elem.tail = i
	143	+ else:
	144	+ if level and (not elem.tail or not elem.tail.strip()):
	145	+ elem.tail = i
	146	+
	147	+ def save(self):
	148	+ if not self.db_name:
	149	+ raise Exception("XML not read from file !")
	150	+
	151	+ self.indent(self.root)
	152	+ self.tree.write(self.db_name, encoding="utf-8")
	153	+
	154	+ def upload_prevention(self, file_name, user, is_super):
	155	+ if not self.file_index.has_key(file_name):
	156	+ return "Filename " + file_name.encode('utf-8') + " not known by server."
	157	+ file_elem = self.file_index[file_name]
	158	+ if self.rejected(file_elem):
	159	+ return "File rejected: " + file_name.encode('utf-8')
	160	+
	161	+ if is_super:
	162	+ ann_elem = file_elem.find("s_ann")
	163	+ if ann_elem is None:
	164	+ return "No superannotator assigned to file " + file_name.encode('utf-8')
	165	+
	166	+ if user != ann_elem.find("annName").text:
	167	+ return "Different superannotator assigned to file " + file_name.encode('utf-8')
	168	+
	169	+ if ann_elem.find("checkinDate") is not None:
	170	+ return "File " + file_name.encode('utf-8') + " already checked in!"
	171	+
	172	+ return None
	173	+
	174	+ else:
	175	+ annotations = file_elem.findall("ann")
	176	+ owners = map(lambda ann: ann.find("annName").text, annotations)
	177	+ if not user in owners:
	178	+ return "User " + user + " doesn't own the file " + file_name.encode('utf-8')
	179	+
	180	+ i = owners.index(user)
	181	+ ann_elem = annotations[i]
	182	+ if ann_elem.find("checkinDate") is not None:
	183	+ return "User " + user + " already checked in the file " + file_name.encode('utf-8')
	184	+
	185	+ return None
	186	+
	187	+ def upload_dest(self, file_name, user):
	188	+ """Upload destination (annName element and ID -- 1 or 2) directory."""
	189	+ file_elem = self.file_index[file_name]
	190	+ annotations = file_elem.findall("ann")
	191	+
	192	+ owners = map(lambda ann: ann.find("annName").text, annotations)
	193	+ exc = Exception("Cannot upload " + file_name.encode('utf-8')
	194	+ + " file by " + user + " annotator !")
	195	+ if not user in owners:
	196	+ raise exc
	197	+
	198	+ i = owners.index(user)
	199	+ ann_elem = annotations[i]
	200	+ if ann_elem.find("checkinDate") is not None:
	201	+ raise exc
	202	+
	203	+ return (ann_elem, i)
	204	+
	205	+ def upload_id(self, file_name, user):
	206	+ return self.upload_dest(file_name, user)[1]
	207	+
	208	+ def return_file(self, file_name, user, reason):
	209	+ (ann_elem, i) = self.upload_dest(file_name, user)
	210	+ ret_elem = ET.SubElement(ann_elem, "return")
	211	+ date_elem = ET.SubElement(ret_elem, "date")
	212	+ date_elem.text = time.strftime("%c")
	213	+ date_elem = ET.SubElement(ret_elem, "reason")
	214	+ if reason is None:
	215	+ reason = ""
	216	+ date_elem.text = reason
	217	+ return i
	218	+
	219	+ def return_file_prim(self, file_name, user, reason):
	220	+ file_elem = self.file_index[file_name]
	221	+ ann_elem = file_elem.find("s_ann")
	222	+
	223	+ if (user != ann_elem.find("annName").text or
	224	+ ann_elem.find("checkinDate") is not None):
	225	+ raise Exception("Cannot upload " + file_name.encode('utf-8')
	226	+ + " file by " + user + " annotator !")
	227	+
	228	+ ret_elem = ET.SubElement(ann_elem, "return")
	229	+ date_elem = ET.SubElement(ret_elem, "date")
	230	+ date_elem.text = time.strftime("%c")
	231	+ date_elem = ET.SubElement(ret_elem, "reason")
	232	+ if reason is None:
	233	+ reason = ""
	234	+ date_elem.text = reason
	235	+
	236	+ def upload(self, file_name, user):
	237	+ (ann_elem, i) = self.upload_dest(file_name, user)
	238	+ date_elem = ET.SubElement(ann_elem, "checkinDate")
	239	+ date_elem.text = time.strftime("%c")
	240	+ return i
	241	+
	242	+ def upload_prim(self, file_name, user):
	243	+ file_elem = self.file_index[file_name]
	244	+ ann_elem = file_elem.find("s_ann")
	245	+
	246	+ if (user != ann_elem.find("annName").text or self.finished(ann_elem)):
	247	+ raise Exception("Cannot upload " + file_name.encode('utf-8')
	248	+ + " file by " + user + " annotator !")
	249	+
	250	+ date_elem = ET.SubElement(ann_elem, "checkinDate")
	251	+ date_elem.text = time.strftime("%c")
	252	+
	253	+ def download(self, file_name, user):
	254	+ file_elem = self.file_index[file_name]
	255	+
	256	+ annotations = file_elem.findall("ann")
	257	+ owners = map(lambda ann: ann.find("annName").text, annotations)
	258	+
	259	+ if self.fixed_for_user(file_elem, user, False):
	260	+ ann, idx = self.ann_elem_for_user(file_elem, user)
	261	+ date = ann.find("return").find("date").text
	262	+ ann.remove(ann.find("return"))
	263	+
	264	+ date_elem = ET.SubElement(ann, "returnDate")
	265	+ date_elem.text = date
	266	+
	267	+ date_elem = ET.SubElement(ann, "checkoutDate")
	268	+ date_elem.text = time.strftime("%c")
	269	+
	270	+ return idx
	271	+
	272	+ else:
	273	+ if user in owners or len(owners) + 1 > self.anno_per_file:
	274	+ raise Exception("Cannot set " + user + " annotator to '"
	275	+ + file_name.encode('utf-8') + "' file !")
	276	+
	277	+ ann_elem = ET.SubElement(file_elem, "ann")
	278	+ ann_name_elem = ET.SubElement(ann_elem, "annName")
	279	+ ann_name_elem.text = user
	280	+
	281	+ date_elem = ET.SubElement(ann_elem, "checkoutDate")
	282	+ date_elem.text = time.strftime("%c")
	283	+
	284	+ return None
	285	+
	286	+ def download_prim(self, file_name, user):
	287	+ file_elem = self.file_index[file_name]
	288	+
	289	+ if self.fixed_for_user(file_elem, user, True):
	290	+ sann = self.sann_elem_for_user(file_elem, user)
	291	+ date = sann.find("return").find("date").text
	292	+ sann.remove(sann.find("return"))
	293	+
	294	+ date_elem = ET.SubElement(sann, "returnDate")
	295	+ date_elem.text = date
	296	+
	297	+ date_elem = ET.SubElement(sann, "checkoutDate")
	298	+ date_elem.text = time.strftime("%c")
	299	+
	300	+ return True
	301	+
	302	+ else:
	303	+ if len(file_elem.findall("s_ann")) > 0:
	304	+ raise Exception("Cannot set " + user + " adjudicator to '"
	305	+ + file_name.encode('utf-8') + "' file !")
	306	+
	307	+ ann_elem = ET.SubElement(file_elem, "s_ann")
	308	+ ann_name_elem = ET.SubElement(ann_elem, "annName")
	309	+ ann_name_elem.text = user
	310	+
	311	+ date_elem = ET.SubElement(ann_elem, "checkoutDate")
	312	+ date_elem.text = time.strftime("%c")
	313	+
	314	+ return False
	315	+
	316	+ def for_annotation(self, user):
	317	+ priority = []
	318	+ result = []
	319	+ for file_elem in self.root:
	320	+ if self.rejected(file_elem) or file_elem.find("s_ann") is not None:
	321	+ continue
	322	+
	323	+ anns = file_elem.findall("ann")
	324	+ owners = map(lambda ann: ann.find("annName").text, anns)
	325	+ if len(owners) < self.anno_per_file and user not in owners:
	326	+ result.append(file_elem.find("name").text)
	327	+
	328	+ if self.fixed_for_user(file_elem, user, False):
	329	+ priority.append(file_elem.find("name").text)
	330	+
	331	+ return priority, result
	332	+
	333	+ def for_adjudication(self, user):
	334	+ priority = []
	335	+ result = []
	336	+ for file_elem in self.root:
	337	+ if self.rejected(file_elem):
	338	+ continue
	339	+
	340	+ anns = file_elem.findall("ann")
	341	+ owners = map(lambda ann: ann.find("annName").text, anns)
	342	+ checked = map(lambda ann: ann.find("checkinDate") != None, anns)
	343	+ if (len(owners) == self.anno_per_file
	344	+ and file_elem.find("s_ann") is None
	345	+ and user not in owners
	346	+ and all(checked)):
	347	+ result.append(file_elem.find("name").text)
	348	+
	349	+ if self.fixed_for_user(file_elem, user, True):
	350	+ priority.append(file_elem.find("name").text)
	351	+
	352	+ return priority, result
	353	+
	354	+ def owns_normal(self, user):
	355	+ return [ file_elem.find("name").text
	356	+ for file_elem
	357	+ in self.ann_owned_index[user] ]
	358	+
	359	+ def owns_super(self, user):
	360	+ return [ file_elem.find("name").text
	361	+ for file_elem
	362	+ in self.adj_owned_index[user] ]
	363	+
	364	+ def owns(self, user):
	365	+ return self.owns_normal(user) + self.owns_super(user)
	366	+
	367	+ def finished_count(self, user):
	368	+ n = 0
	369	+ for file_elem in self.root:
	370	+
	371	+ if self.rejected(file_elem):
	372	+ continue
	373	+
	374	+ items = file_elem.findall("ann")
	375	+ items.extend(file_elem.findall("s_ann"))
	376	+ for ann_elem in items:
	377	+ ann_name = ann_elem.find("annName").text
	378	+ if ann_name == user and self.finished(ann_elem) and not self.returned(ann_elem):
	379	+ n += 1
	380	+
	381	+ return n
...	...

gui/dfs/msg/__init__.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/dfs/msg/__init__.py
...	...

gui/dfs/msg/message.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/dfs/msg/message.py
	1	+# -- coding: utf-8 --
	2	+__all__ = ["decode", "Message", "OkMessage", "KoMessage", "DownloadRequest"
	3	+ , "UploadRequest", "DownloadPrimRequest", "ClientDone", "ServerDone"
	4	+ , "NumMessage", "BoolMessage", "ReturnRequest", "CheckoutRequest",
	5	+ "CheckinRequest", "StatsRequest"]
	6	+
	7	+
	8	+class Message(object):
	9	+
	10	+ """
	11	+ Base class for SSL communication messages.
	12	+ """
	13	+
	14	+ def __init__(self, contents=""):
	15	+ self.contents = contents
	16	+
	17	+ def get_contents(self):
	18	+ return self.contents
	19	+
	20	+ @classmethod
	21	+ def from_contents(cls, contents):
	22	+ return cls(contents)
	23	+
	24	+ def encode(self):
	25	+ """
	26	+ Prepare message to be sent over network.
	27	+ WARNING: Do not override this method.
	28	+ """
	29	+ # Add "-" on the beggining to prevent
	30	+ # message contents from being void.
	31	+ return self.msg_type, "-" + self.get_contents()
	32	+
	33	+
	34	+class NumMessage(Message):
	35	+
	36	+ def __init__(self, n):
	37	+ self.number = n
	38	+
	39	+ def get_contents(self):
	40	+ return str(self.number)
	41	+
	42	+ @classmethod
	43	+ def from_contents(cls, contents):
	44	+ return cls(int(contents))
	45	+
	46	+ def get_number(self):
	47	+ return self.number
	48	+
	49	+
	50	+class BoolMessage(Message):
	51	+
	52	+ def __init__(self, b):
	53	+ self.b = b
	54	+
	55	+ def get_contents(self):
	56	+ return "1" if self.b else "0"
	57	+
	58	+ @classmethod
	59	+ def from_contents(cls, contents):
	60	+ return cls(bool(int(contents)))
	61	+
	62	+ def get_boolean(self):
	63	+ return self.b
	64	+
	65	+class ReturnRequest(NumMessage):
	66	+ pass
	67	+
	68	+class CheckoutRequest(Message):
	69	+ pass
	70	+
	71	+class CheckinRequest(NumMessage):
	72	+ pass
	73	+
	74	+class DownloadRequest(NumMessage):
	75	+ pass
	76	+
	77	+class DownloadPrimRequest(NumMessage):
	78	+ pass
	79	+
	80	+class UploadRequest(NumMessage):
	81	+ pass
	82	+
	83	+class OkMessage(Message):
	84	+ pass
	85	+
	86	+class KoMessage(Message):
	87	+ pass
	88	+
	89	+class StatsRequest(Message):
	90	+ pass
	91	+
	92	+class ServerDone(Message):
	93	+ pass
	94	+
	95	+class ClientDone(Message):
	96	+ pass
	97	+
	98	+
	99	+def subclasses(cls):
	100	+ yield cls
	101	+ for child in cls.__subclasses__():
	102	+ for desc in subclasses(child):
	103	+ yield desc
	104	+
	105	+def decode(msg_type, contents):
	106	+ for cls in subclasses(Message):
	107	+ if cls.msg_type == msg_type:
	108	+ # Discard first, dummy character;
	109	+ # Confront Message.encode method.
	110	+ return cls.from_contents(contents[1:])
	111	+
	112	+for i, cls in enumerate(subclasses(Message)):
	113	+ cls.msg_type = i
...	...

gui/dfs/msg/stream.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/dfs/msg/stream.py
	1	+# -- coding: utf-8 --
	2	+import struct
	3	+
	4	+from message import decode
	5	+
	6	+__all__ = ["read_msg", "write_msg", "EndOfStream"]
	7	+
	8	+class EndOfStream(Exception):
	9	+ pass
	10	+
	11	+class BadMessage(Exception):
	12	+ pass
	13	+
	14	+def read_msg(connstream):
	15	+ encoded = read_encoded(connstream)
	16	+ return decode(*encoded)
	17	+
	18	+def accept_msg(connstream, cls):
	19	+ msg = read_msg(connstream)
	20	+ if type(msg) != cls:
	21	+ raise BadMessage("Expected %s, got %s" %
	22	+ (cls.__name__, type(msg).__name__))
	23	+ return msg
	24	+
	25	+def write_msg(connstream, msg):
	26	+ encoded = msg.encode()
	27	+ write_encoded(connstream, *encoded)
	28	+
	29	+def read_encoded(connstream):
	30	+ _type = read_type(connstream)
	31	+ length = read_length(connstream)
	32	+ content = read_content(connstream, length)
	33	+ return _type, content
	34	+
	35	+def write_encoded(connstream, _type, content):
	36	+ write_type(connstream, _type)
	37	+ write_length(connstream, len(content))
	38	+ write_content(connstream, content)
	39	+
	40	+def read_type(connstream):
	41	+ data = read_all(connstream, 1)
	42	+ return struct.unpack("!B", data)[0]
	43	+
	44	+def write_type(connstream, _type):
	45	+ data = struct.pack("!B", _type)
	46	+ write_all(connstream, data)
	47	+
	48	+def read_length(connstream):
	49	+ data = read_all(connstream, 4)
	50	+ return struct.unpack("!L", data)[0]
	51	+
	52	+def write_length(connstream, n):
	53	+ data = struct.pack("!L", n)
	54	+ write_all(connstream, data)
	55	+
	56	+def read_content(connstream, length):
	57	+ return read_all(connstream, length)
	58	+
	59	+def write_content(connstream, content):
	60	+ write_all(connstream, content)
	61	+
	62	+def write_all(connstream, data):
	63	+ k = connstream.write(data)
	64	+ if k != len(data):
	65	+ raise Exception("Did not send all data !")
	66	+
	67	+def read_all(connstream, k):
	68	+ data = ""
	69	+ while k > 0:
	70	+ # part = connstream.read()
	71	+ part = connstream.read(min(k, 1024))
	72	+ if len(part) == 0:
	73	+ raise EndOfStream
	74	+ k -= len(part)
	75	+ # if k < 0:
	76	+ # raise Exception("Received more than expected !")
	77	+ data += part
	78	+ return data
	79	+
...	...

gui/dfs/repo.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/dfs/repo.py
	1	+# -- coding: utf-8 --
	2	+import os
	3	+import shutil
	4	+import pysvn
	5	+
	6	+class Repo:
	7	+
	8	+ def __init__(self, path, login, passwd):
	9	+ self.path = path
	10	+ def get_login(_realm, _username, _may_save):
	11	+ return True, login, passwd, False
	12	+ self.svn = pysvn.Client()
	13	+ self.svn.callback_get_login = get_login
	14	+ self.svn.update(self.path)
	15	+
	16	+ def add(self, src_path):
	17	+ part_name = os.path.basename(src_path)
	18	+ dest_path = os.path.join(self.new_path(), part_name)
	19	+
	20	+ if (not self.svn.info(dest_path) is None) or (os.path.exists(dest_path)):
	21	+ raise Exception(src_path+" already present in repository.")
	22	+
	23	+ shutil.copy(src_path, dest_path)
	24	+ self.svn.add(dest_path)
	25	+
	26	+ def remove(self, filename, anno_per_file):
	27	+ paths = set()
	28	+ paths.add(os.path.join(self.new_path(), filename))
	29	+ for i in range(anno_per_file):
	30	+ paths.add(self.upload_path(filename, i))
	31	+ paths.add(os.path.join(self.finito_path(), filename))
	32	+
	33	+ removed = []
	34	+ for path in paths:
	35	+ if os.path.exists(path):
	36	+ self.svn.remove(path)
	37	+ removed.append(path)
	38	+
	39	+ return removed
	40	+
	41	+ def db_path(self):
	42	+ return os.path.join(self.path, "db.xml")
	43	+
	44	+ def new_path(self):
	45	+ return os.path.join(self.path, "new")
	46	+
	47	+ def ann_path(self, idx):
	48	+ ids = chr(idx+ord('A'))
	49	+ tail = os.path.join("annotation", ids)
	50	+ return os.path.join(self.path, tail)
	51	+
	52	+ def finito_path(self):
	53	+ return os.path.join(self.path, "adjudication")
	54	+
	55	+ def upload_path(self, file_name, idx):
	56	+ return os.path.join(self.ann_path(idx), file_name)
	57	+
	58	+ def upload_prim_path(self, file_name):
	59	+ return os.path.join(self.finito_path(), file_name)
	60	+
	61	+ def read_data(self, path):
	62	+ with open(path) as src:
	63	+ return src.read()
	64	+
	65	+ def read_contents(self, path, exts):
	66	+ contents = {}
	67	+ for ext in exts:
	68	+ contents[ext] = self.read_data(path + ext)
	69	+ return contents
	70	+
	71	+ def write_data(self, path, data):
	72	+ with open(path, "w") as dest:
	73	+ dest.write(data)
	74	+
	75	+ def write_contents(self, path, contents):
	76	+ for ext, data in contents.iteritems():
	77	+ dest_path = path + ext
	78	+ self.write_data(dest_path, data)
	79	+ if self.svn.info(dest_path) is None:
	80	+ self.svn.add(dest_path)
	81	+
	82	+ def upload(self, file_id, idx, contents):
	83	+ self.write_contents(self.upload_path(file_id, idx), contents)
	84	+
	85	+ def upload_prim(self, file_id, contents):
	86	+ self.write_contents(self.upload_prim_path(file_id), contents)
	87	+
	88	+ def download(self, file_id, exts):
	89	+ return self.read_contents(os.path.join(self.new_path(), file_id), exts)
	90	+
	91	+ def download_prim(self, file_id, exts, anno_per_file):
	92	+ result = []
	93	+ for i in range(anno_per_file):
	94	+ conts = self.read_contents(os.path.join(self.ann_path(i), file_id), exts)
	95	+ result.append(conts)
	96	+ return result
	97	+
	98	+ def checkout(self, file_id, idx, exts):
	99	+ src_paths = [ self.upload_path(file_id, idx)
	100	+ , os.path.join(self.new_path(), file_id) ]
	101	+ for path in src_paths:
	102	+ if all(os.path.exists(path + ext) for ext in exts):
	103	+ return self.read_contents(path, exts)
	104	+
	105	+ def checkout_prim(self, file_id, exts):
	106	+ path = os.path.join(self.finito_path(), file_id)
	107	+ if all(os.path.exists(path + ext) for ext in exts):
	108	+ return self.read_contents(path, exts)
	109	+
	110	+ def commit(self, message):
	111	+ self.svn.checkin(self.path, message, recurse=True)
	112	+
	113	+ def revert(self):
	114	+ self.svn.revert(self.path, recurse=True)
...	...

gui/dfs/server.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/dfs/server.py
	1	+# -- coding: utf-8 --
	2	+import sys
	3	+import os
	4	+import traceback
	5	+import socket, ssl
	6	+import random
	7	+from optparse import OptionParser
	8	+import signal
	9	+
	10	+from utils import validate_user, UserInvalid, send_contents, receive_contents
	11	+from msg.stream import read_msg, write_msg, accept_msg, EndOfStream, BadMessage
	12	+from msg.message import *
	13	+from repo import Repo
	14	+from database import Database
	15	+from config import Config
	16	+
	17	+class SSLServer:
	18	+
	19	+ def __init__(self, host, port, backlog, cert_file, key_file,
	20	+ repository, svn_login, svn_passwd, pass_file,
	21	+ users_file, file_exts, anno_per_file, log=None):
	22	+ """
	23	+ Initialize SSL server.
	24	+
	25	+ params:
	26	+ =======
	27	+ host : str
	28	+ Host name.
	29	+ port : int
	30	+ Port number.
	31	+ backlog : int
	32	+ Maximum number of waiting connections.
	33	+ cert_file : path
	34	+ Public PEM certificate.
	35	+ key_file : path
	36	+ Private key.
	37	+ pass_file : path
	38	+ File with client passwords.
	39	+ users_file : path
	40	+ File with additional users configuration.
	41	+ repository : path
	42	+ Subversion working copy.
	43	+ svn_login : str
	44	+ Subversion login.
	45	+ svn_passwd : str
	46	+ Subversion password.
	47	+ file_exts : [str]
	48	+ List of file extensions.
	49	+ anno_per_file : int
	50	+ Number of normal annotators per file (1 or 2)
	51	+ """
	52	+ self.cert_file = cert_file
	53	+ self.key_file = key_file
	54	+ self.pass_file = pass_file
	55	+ self.users_file = users_file
	56	+ self.wc = Repo(repository, svn_login, svn_passwd)
	57	+ self.file_exts = file_exts
	58	+ if log is not None:
	59	+ self.log = open(log, "a")
	60	+ else:
	61	+ self.log = sys.stdout
	62	+ self.bound = bind_socket(host, port, backlog=backlog)
	63	+ self.serving = None
	64	+ self.anno_per_file = anno_per_file
	65	+
	66	+ def run(self):
	67	+ while True:
	68	+ newsocket, fromaddr = self.bound.accept()
	69	+ self.serving = fromaddr
	70	+ connstream = None
	71	+ try:
	72	+ connstream = ssl.wrap_socket(newsocket,
	73	+ server_side=True,
	74	+ certfile=self.cert_file,
	75	+ keyfile=self.key_file,
	76	+ ssl_version=ssl.PROTOCOL_TLSv1)
	77	+
	78	+ login = validate_user(connstream, self.pass_file)
	79	+ self.serve_client(connstream, login)
	80	+ except UserInvalid as login:
	81	+ print >> self.log, "AUTH ERROR:", login
	82	+ except BadMessage as info:
	83	+ print >> self.log, "BAD MESSAGE ERROR:", info
	84	+ print >> self.log, traceback.format_exc().strip()
	85	+ except EndOfStream:
	86	+ print >> self.log, "UNEXPECTED END OF STREAM:"
	87	+ print >> self.log, traceback.format_exc().strip()
	88	+ except:
	89	+ print >> self.log, "UNEXPECTED ERROR:"
	90	+ print >> self.log, traceback.format_exc().strip()
	91	+ finally:
	92	+ try:
	93	+ if not connstream is None:
	94	+ connstream.shutdown(socket.SHUT_RDWR)
	95	+ except:
	96	+ print >> self.log, "UNEXPECTED ERROR:"
	97	+ print >> self.log, traceback.format_exc().strip()
	98	+ finally:
	99	+ if not connstream is None:
	100	+ connstream.close()
	101	+ self.serving = None
	102	+
	103	+ def exit(self):
	104	+ if self.serving is not None:
	105	+ print ("Client from %s connected to the server"
	106	+ % str(self.serving))
	107	+ else:
	108	+ sys.exit(0)
	109	+
	110	+ def process_msg(self, msg, stream, login):
	111	+ if type(msg) == UploadRequest:
	112	+ self.process_upload_msg(msg, stream, login, checkin=False)
	113	+ elif type(msg) == CheckinRequest:
	114	+ self.process_upload_msg(msg, stream, login, checkin=True)
	115	+ elif type(msg) == CheckoutRequest:
	116	+ self.process_checkout_msg(msg, stream, login)
	117	+ elif type(msg) == DownloadRequest:
	118	+ self.process_download_msg(msg, stream, login)
	119	+ elif type(msg) == DownloadPrimRequest:
	120	+ self.process_download_prim_msg(msg, stream, login)
	121	+ elif type(msg) == ReturnRequest:
	122	+ self.process_return_msg(msg, stream, login)
	123	+ elif type(msg) == StatsRequest:
	124	+ self.process_stats_msg(msg, stream, login)
	125	+ else:
	126	+ print >> self.log, login, "sent:", msg
	127	+
	128	+ def process_return_msg(self, msg, stream, login):
	129	+ reason = accept_msg(stream, Message).get_contents().decode("utf-8")
	130	+
	131	+ usr_cfg = Config(self.users_file)
	132	+ try:
	133	+ user_adj = login in usr_cfg["auth.adjudicators"].split()
	134	+ except KeyError:
	135	+ user_adj = False
	136	+
	137	+ db = Database(self.wc.db_path(), self.anno_per_file)
	138	+ n = msg.get_number()
	139	+ for _ in range(n):
	140	+ file_id = accept_msg(stream, Message).get_contents()
	141	+ is_adj = accept_msg(stream, BoolMessage).get_boolean()
	142	+ if is_adj and not user_adj:
	143	+ write_msg(stream, KoMessage())
	144	+ print >> self.log, "Upload error - user " + login + " tried to upload super annotated file " + file_id + " but is not a super annotator"
	145	+ continue
	146	+ else:
	147	+ error = db.upload_prevention(file_id, login, is_adj)
	148	+ if error is not None:
	149	+ write_msg(stream, KoMessage())
	150	+ print >> self.log, "Upload error by user " + login + ". Details: "+error
	151	+ continue
	152	+
	153	+ write_msg(stream, OkMessage())
	154	+ contents = receive_contents(stream)
	155	+ for key in contents.keys():
	156	+ if key not in self.file_exts:
	157	+ print >> self.log, "Skipped file uploaded by user " + login + ". Filename:"+file_id+key
	158	+ del contents[key]
	159	+
	160	+ if is_adj is False:
	161	+ idx = db.return_file(file_id, login, reason)
	162	+ self.wc.upload(file_id, idx, contents)
	163	+ else:
	164	+ db.return_file_prim(file_id, login, reason)
	165	+ self.wc.upload_prim(file_id, contents)
	166	+
	167	+ db.save()
	168	+ accept_msg(stream, ClientDone)
	169	+ self.wc.commit("return request from %s"
	170	+ % (login))
	171	+ write_msg(stream, ServerDone())
	172	+
	173	+
	174	+ def process_upload_msg(self, msg, stream, login, checkin=False):
	175	+ usr_cfg = Config(self.users_file)
	176	+ try:
	177	+ user_adj = login in usr_cfg["auth.adjudicators"].split()
	178	+ except KeyError:
	179	+ user_adj = False
	180	+
	181	+ db = Database(self.wc.db_path(), self.anno_per_file)
	182	+ n = msg.get_number()
	183	+ for _ in range(n):
	184	+ file_id = accept_msg(stream, Message).get_contents()
	185	+ is_adj = accept_msg(stream, BoolMessage).get_boolean()
	186	+ if is_adj and not user_adj:
	187	+ write_msg(stream, KoMessage())
	188	+ print >> self.log, "Upload error - user " + login + " tried to upload super annotated file " + file_id + " but is not a super annotator"
	189	+ continue
	190	+ else:
	191	+ error = db.upload_prevention(file_id, login, is_adj)
	192	+ if error is not None:
	193	+ write_msg(stream, KoMessage())
	194	+ print >> self.log, "Upload error by user " + login + ". Details: "+error
	195	+ continue
	196	+
	197	+ write_msg(stream, OkMessage())
	198	+ contents = receive_contents(stream)
	199	+ for key in contents.keys():
	200	+ if key not in self.file_exts:
	201	+ print >> self.log, "Skipped file uploaded by user " + login + ". Filename:"+file_id+key
	202	+ del contents[key]
	203	+
	204	+ if is_adj is False:
	205	+ if checkin:
	206	+ idx = db.upload(file_id, login)
	207	+ else:
	208	+ idx = db.upload_id(file_id, login)
	209	+ self.wc.upload(file_id, idx, contents)
	210	+ else:
	211	+ if checkin:
	212	+ db.upload_prim(file_id, login)
	213	+ self.wc.upload_prim(file_id, contents)
	214	+
	215	+ if checkin:
	216	+ db.save()
	217	+ accept_msg(stream, ClientDone)
	218	+ self.wc.commit("%s request from %s"
	219	+ % ("checkin" if checkin else "upload", login))
	220	+ write_msg(stream, ServerDone())
	221	+
	222	+ def process_checkout_msg(self, msg, stream, login):
	223	+ db = Database(self.wc.db_path(), self.anno_per_file)
	224	+
	225	+ ann_files = db.owns_normal(login)
	226	+ write_msg(stream, NumMessage(len(ann_files)))
	227	+ for file_id in ann_files:
	228	+ write_msg(stream, Message(file_id))
	229	+ idx = db.upload_id(file_id, login)
	230	+ contents = self.wc.checkout(file_id, idx, self.file_exts)
	231	+ send_contents(stream, contents)
	232	+
	233	+ adj_files = db.owns_super(login)
	234	+ write_msg(stream, NumMessage(len(adj_files)))
	235	+ write_msg(stream, NumMessage(self.anno_per_file))
	236	+ for file_id in adj_files:
	237	+ write_msg(stream, Message(file_id))
	238	+
	239	+ contents = self.wc.download_prim(file_id, self.file_exts, self.anno_per_file)
	240	+ for conts in contents:
	241	+ send_contents(stream, conts)
	242	+
	243	+ conts3 = self.wc.checkout_prim(file_id, self.file_exts)
	244	+ if conts3 is None:
	245	+ write_msg(stream, BoolMessage(False))
	246	+ else:
	247	+ write_msg(stream, BoolMessage(True))
	248	+ send_contents(stream, conts3)
	249	+
	250	+ accept_msg(stream, ClientDone)
	251	+ #self.wc.commit("checkout request from %s" % login)
	252	+ write_msg(stream, ServerDone())
	253	+
	254	+ def process_stats_msg(self, msg, stream, login):
	255	+ usr_cfg = Config(self.users_file)
	256	+ db = Database(self.wc.db_path(), self.anno_per_file)
	257	+
	258	+ write_msg(stream, NumMessage(db.finished_count(login)))
	259	+
	260	+ accept_msg(stream, ClientDone)
	261	+ write_msg(stream, ServerDone())
	262	+
	263	+ def process_download_msg(self, msg, stream, login):
	264	+ usr_cfg = Config(self.users_file)
	265	+ db = Database(self.wc.db_path(), self.anno_per_file)
	266	+
	267	+ down_num = msg.get_number()
	268	+ owns_num = len(db.owns_normal(login))
	269	+ limit = float('inf')
	270	+ try:
	271	+ limit = int(usr_cfg["limits.%s.annotation" % login])
	272	+ except KeyError:
	273	+ limit = int(usr_cfg["limits.annotation"])
	274	+ if down_num + owns_num > limit:
	275	+ down_num = max(0, limit - owns_num)
	276	+
	277	+ for_ann_fixed, for_ann = db.for_annotation(login)
	278	+ if len(for_ann) + len(for_ann_fixed) == 0:
	279	+ print >> self.log, "User " + login + " has no more files to download for annotation."
	280	+
	281	+ down_files = sample(for_ann_fixed, down_num)
	282	+ down_num = max(0, down_num - len(down_files))
	283	+ down_files = down_files + sample(for_ann, down_num)
	284	+
	285	+ write_msg(stream, NumMessage(len(down_files)))
	286	+
	287	+ for file_id in down_files:
	288	+ fixed_idx = db.download(file_id, login)
	289	+ write_msg(stream, Message(file_id))
	290	+ if fixed_idx is None:
	291	+ contents = self.wc.download(file_id, self.file_exts)
	292	+ else:
	293	+ contents = self.wc.checkout(file_id, fixed_idx, self.file_exts)
	294	+ send_contents(stream, contents)
	295	+ db.save()
	296	+ accept_msg(stream, ClientDone)
	297	+ self.wc.commit("download request from %s" % login)
	298	+ write_msg(stream, ServerDone())
	299	+
	300	+ def process_download_prim_msg(self, msg, stream, login):
	301	+ usr_cfg = Config(self.users_file)
	302	+ try:
	303	+ adjudicators = usr_cfg["auth.adjudicators"].split()
	304	+ except KeyError:
	305	+ adjudicators = []
	306	+ if login in adjudicators:
	307	+ write_msg(stream, OkMessage())
	308	+ else:
	309	+ write_msg(stream, KoMessage())
	310	+ return
	311	+ db = Database(self.wc.db_path(), self.anno_per_file)
	312	+
	313	+ down_num = msg.get_number()
	314	+ owns_num = len(db.owns_super(login))
	315	+ limit = float('inf')
	316	+ try:
	317	+ limit = int(usr_cfg["limits.%s.adjudication" % login])
	318	+ except KeyError:
	319	+ limit = int(usr_cfg["limits.adjudication"])
	320	+ if down_num + owns_num > limit:
	321	+ down_num = max(0, limit - owns_num)
	322	+
	323	+ for_adj_fixed, for_adj = db.for_adjudication(login)
	324	+ if len(for_adj) + len(for_adj_fixed) == 0:
	325	+ print >> self.log, "User " + login + " has no more files to download for superannotation."
	326	+
	327	+ down_files = sample(for_adj_fixed, down_num)
	328	+ down_num = max(0, down_num - len(down_files))
	329	+ down_files = down_files + sample(for_adj, down_num)
	330	+
	331	+ write_msg(stream, NumMessage(len(down_files)))
	332	+ write_msg(stream, NumMessage(self.anno_per_file))
	333	+ for file_id in down_files:
	334	+ write_msg(stream, Message(file_id))
	335	+ fixed = db.download_prim(file_id, login)
	336	+
	337	+ contents = self.wc.download_prim(file_id, self.file_exts, self.anno_per_file)
	338	+ for conts in contents:
	339	+ send_contents(stream, conts)
	340	+
	341	+ if fixed:
	342	+ write_msg(stream, BoolMessage(True))
	343	+ contents = self.wc.checkout_prim(file_id, self.file_exts)
	344	+ send_contents(stream, contents)
	345	+ else:
	346	+ write_msg(stream, BoolMessage(False))
	347	+
	348	+ db.save()
	349	+ accept_msg(stream, ClientDone)
	350	+ self.wc.commit("download' request from %s" % login)
	351	+ write_msg(stream, ServerDone())
	352	+
	353	+ def serve_client(self, connstream, login):
	354	+ msg = read_msg(connstream)
	355	+ try:
	356	+ self.process_msg(msg, connstream, login)
	357	+ except:
	358	+ self.wc.revert()
	359	+ raise
	360	+
	361	+def bind_socket(host, port, backlog=5):
	362	+ bound = socket.socket()
	363	+ bound.bind((host, port))
	364	+ bound.listen(backlog)
	365	+ return bound
	366	+
	367	+def sample(population, n):
	368	+ if len(population) > n:
	369	+ return random.sample(population, n)
	370	+ else:
	371	+ return population
	372	+
	373	+if __name__ == "__main__":
	374	+ optparser = OptionParser(usage="""usage: %prog CONFIG""")
	375	+ (options, args) = optparser.parse_args()
	376	+ if len(args) != 1:
	377	+ optparser.print_help()
	378	+ sys.exit(0)
	379	+
	380	+ cfg = Config(args[0])
	381	+ server = SSLServer(
	382	+ host=cfg["connection.host"],
	383	+ port=int(cfg["connection.port"]),
	384	+ backlog=int(cfg["connection.backlog"]),
	385	+ cert_file=cfg["connection.certfile"],
	386	+ key_file=cfg["connection.keyfile"],
	387	+ repository=cfg["svn.repository"],
	388	+ svn_login=cfg["svn.login"],
	389	+ svn_passwd=cfg["svn.passwd"],
	390	+ pass_file=cfg["users.passfile"],
	391	+ users_file=cfg["users.config"],
	392	+ file_exts=cfg["file_extensions"].split(),
	393	+ anno_per_file=int(cfg["anno_per_file"])
	394	+ )
	395	+
	396	+ def handler(signum, frame):
	397	+ server.exit()
	398	+
	399	+ signal.signal(signal.SIGINT, handler)
	400	+
	401	+ server.run()
...	...

gui/dfs/utils.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/dfs/utils.py
	1	+# -- coding: utf-8 --
	2	+import hashlib
	3	+
	4	+from msg.stream import accept_msg, write_msg
	5	+from msg.message import Message, OkMessage, KoMessage, NumMessage
	6	+
	7	+class UserInvalid(Exception):
	8	+ pass
	9	+
	10	+def validate_user(conns, pass_file):
	11	+ pass_dict = parse_passwd(pass_file)
	12	+ login = accept_msg(conns, Message).get_contents()
	13	+ passwd = accept_msg(conns, Message).get_contents()
	14	+ if pass_dict.has_key(login) and pass_dict[login] == passwd:
	15	+ write_msg(conns, OkMessage())
	16	+ return login
	17	+ else:
	18	+ write_msg(conns, KoMessage())
	19	+ raise UserInvalid(login)
	20	+
	21	+def parse_passwd(pass_file):
	22	+ result = {}
	23	+ with open(pass_file) as f:
	24	+ for line in f:
	25	+ login, passwd = line.split("=")
	26	+ login = login.strip()
	27	+ passwd = passwd.strip()
	28	+ result[login] = passwd
	29	+ return result
	30	+
	31	+def _check_sum(*args):
	32	+ m = hashlib.sha256()
	33	+ for arg in args:
	34	+ m.update(arg)
	35	+ return m
	36	+
	37	+def check_sum(*args):
	38	+ return _check_sum(*args).digest()
	39	+
	40	+def check_hexsum(*args):
	41	+ return _check_sum(*args).hexdigest()
	42	+
	43	+def send_contents(stream, contents):
	44	+ write_msg(stream, NumMessage(len(contents)))
	45	+ for ext, data in contents.iteritems():
	46	+ write_msg(stream, Message(ext))
	47	+ write_msg(stream, Message(data))
	48	+ write_msg(stream, Message(check_sum(ext, data)))
	49	+ accept_msg(stream, OkMessage)
	50	+
	51	+def receive_contents(stream):
	52	+ contents = {}
	53	+ n = accept_msg(stream, NumMessage).get_number()
	54	+ for _ in range(n):
	55	+ ext = accept_msg(stream, Message).get_contents()
	56	+ data = accept_msg(stream, Message).get_contents()
	57	+ csum = accept_msg(stream, Message).get_contents()
	58	+ if csum == check_sum(ext, data):
	59	+ write_msg(stream, OkMessage())
	60	+ else:
	61	+ write_msg(stream, KoMessage())
	62	+ contents[ext] = data
	63	+ return contents
...	...

gui/dir_cache.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/dir_cache.py
	1	+# -- coding: utf-8 --
	2	+"""Directory cache manipulation"""
	3	+
	4	+import os
	5	+
	6	+class DirCache:
	7	+
	8	+ def __init__(self, path, size):
	9	+ if not os.path.exists(path):
	10	+ with open(path, "w") as f: pass
	11	+ self.path = path
	12	+ self.size = size
	13	+
	14	+ def add(self, dir_name):
	15	+ cache = self.get()
	16	+ if dir_name in cache:
	17	+ cache.remove(dir_name)
	18	+ cache.insert(0, dir_name)
	19	+ self.write(cache[:self.size])
	20	+
	21	+ def get(self):
	22	+ with open(self.path) as f:
	23	+ return [x.strip() for x in f.readlines()]
	24	+
	25	+ def write(self, cache):
	26	+ with open(self.path, "w") as f:
	27	+ for entry in cache:
	28	+ print >> f, entry
...	...

gui/footprints.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/footprints.py
	1	+# -- coding: utf-8 --
	2	+import os
	3	+import copy
	4	+
	5	+# from dfs.utils import check_hexsum as check_sum
	6	+
	7	+def getmtime(path):
	8	+ return round(os.path.getmtime(path))
	9	+
	10	+from utils import all_files
	11	+
	12	+class FootPrints:
	13	+
	14	+ """Class for footprints file manipulation."""
	15	+
	16	+ def __init__(self, prints_path, dir_path, exts):
	17	+ self.prints_path = prints_path
	18	+ self.dir_path = dir_path
	19	+ self.exts = exts
	20	+ if not os.path.exists(prints_path):
	21	+ with open(prints_path, "w") as f: pass
	22	+ self.prints = self.read()
	23	+ self.backup = copy.deepcopy(self.prints)
	24	+
	25	+ def modified(self, name):
	26	+ try:
	27	+ # (mtime, csum) = self.get_print(name)
	28	+ mtime = self.get_print(name)
	29	+ except KeyError:
	30	+ return False
	31	+
	32	+ path = os.path.join(self.dir_path, name)
	33	+ if mtime != self.print_now(path):
	34	+ return True
	35	+ return False
	36	+
	37	+ """ sprawdza czy nie pojawily sie jakies nowe pliki, ewentualnie dodajac dla nich wpisy """
	38	+ def refresh(self):
	39	+ old_prints = self.prints
	40	+ new_prints = {}
	41	+ for name in all_files(self.dir_path, self.exts):
	42	+ if old_prints.has_key(name):
	43	+ new_prints[name] = old_prints[name]
	44	+ else:
	45	+ path = os.path.join(self.dir_path, name)
	46	+ new_prints[name] = self.print_now(path)
	47	+ self.prints = new_prints
	48	+
	49	+ def renew(self, name):
	50	+ path = os.path.join(self.dir_path, name)
	51	+ try:
	52	+ self.prints[name] = self.print_now(path)
	53	+ except:
	54	+ del self.prints[name]
	55	+ print "Footprints: "+ path + " file not found."
	56	+
	57	+ def renew_all(self):
	58	+ for key in self.prints.keys():
	59	+ self.renew(key)
	60	+
	61	+ def get_print(self, name):
	62	+ path = os.path.join(self.dir_path, name)
	63	+ return self.prints[name]
	64	+
	65	+ def print_now(self, path):
	66	+ return max(getmtime(path + ext) for ext in self.exts)
	67	+
	68	+ def read(self):
	69	+ prints = {}
	70	+ with open(self.prints_path) as f:
	71	+ for line in f:
	72	+ key, mtime = line.split()
	73	+ prints[key] = float(mtime)
	74	+ return prints
	75	+
	76	+ def write(self):
	77	+ with open(self.prints_path, "w") as f:
	78	+ # for key, (mtime, csum) in sorted(self.prints.items()):
	79	+ # print >> f, key, "=", mtime, csum
	80	+ for key, mtime in sorted(self.prints.items()):
	81	+ print >> f, key, mtime
	82	+
	83	+ def save(self):
	84	+ if self.prints != self.backup:
	85	+ self.write()
...	...

gui/i18n.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/i18n.py
	1	+# -- coding: utf-8 --
	2	+import os, sys
	3	+import locale
	4	+import gettext
	5	+import const
	6	+
	7	+def get_path():
	8	+ '''Get the path to this script no matter how it's run.'''
	9	+ #Determine if the application is a py/pyw or a frozen exe.
	10	+ if hasattr(sys, 'frozen'):
	11	+ # If run from exe
	12	+ dir_path = os.path.dirname(sys.executable)
	13	+ elif '__file__' in locals():
	14	+ # If run from py
	15	+ dir_path = os.path.dirname(__file__)
	16	+ else:
	17	+ # If run from command line
	18	+ dir_path = sys.path[0]
	19	+ return dir_path
	20	+
	21	+
	22	+# Change this variable to your app name!
	23	+# The translation files will be under
	24	+# @LOCALE_DIR@/@LANGUAGE@/LC_MESSAGES/@APP_NAME@.mo
	25	+#
	26	+APP_NAME = "manager"
	27	+# This is ok for maemo. Not sure in a regular desktop:
	28	+APP_DIR = get_path()
	29	+LOCALE_DIR = os.path.join(APP_DIR, 'po')
	30	+
	31	+langs = ["pl"]
	32	+if (const.lang == "en"):
	33	+ langs = ["en"]
	34	+
	35	+# Lets tell those details to gettext
	36	+gettext.install (True)
	37	+gettext.bindtextdomain (APP_NAME,
	38	+ LOCALE_DIR)
	39	+gettext.textdomain (APP_NAME)
	40	+language = gettext.translation (APP_NAME,
	41	+ LOCALE_DIR,
	42	+ languages = langs,
	43	+ fallback = False)
	44	+
	45	+
0	46	\ No newline at end of file
...	...

gui/img/down.png 0 → 100644

View file @44cbbee

846 Bytes

gui/img/ready.png 0 → 100644

View file @44cbbee

849 Bytes

gui/img/textimg_1.png 0 → 100644

View file @44cbbee

522 Bytes

gui/img/textimg_2.png 0 → 100644

View file @44cbbee

648 Bytes

gui/img/textimg_3.png 0 → 100644

View file @44cbbee

782 Bytes

gui/img/up.png 0 → 100644

View file @44cbbee

837 Bytes

gui/log_thread.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/log_thread.py
	1	+# -- coding: utf-8 --
	2	+import wx
	3	+from Queue import Queue
	4	+from threading import Thread
	5	+import copy
	6	+
	7	+class RThread(Thread):
	8	+
	9	+ def __init__(self, target, args, kwargs):
	10	+ self.target = target
	11	+ self.args = args
	12	+ self.kwargs = kwargs
	13	+ self.result = None
	14	+ Thread.__init__(self)
	15	+
	16	+ def run(self):
	17	+ self.result = self.target(self.args, *self.kwargs)
	18	+
	19	+class LogThread(Thread):
	20	+
	21	+ def __init__(self, text_ctrl, target, on_end=None, args=(), kwargs={}):
	22	+ self.text_ctrl = text_ctrl
	23	+ self.target = target
	24	+ self.on_end = on_end
	25	+ self.args = args
	26	+ self.kwargs = kwargs
	27	+ Thread.__init__(self)
	28	+
	29	+ def run(self):
	30	+ queue = Queue()
	31	+ kwargs = copy.copy(self.kwargs)
	32	+ kwargs["log"] = queue
	33	+ worker = RThread( target=self.target
	34	+ , args=self.args
	35	+ , kwargs=kwargs )
	36	+ worker.start()
	37	+ while True:
	38	+ msg = queue.get()
	39	+ if msg is None:
	40	+ break
	41	+ wx.CallAfter(self.print_msg, msg)
	42	+ worker.join()
	43	+
	44	+ if self.on_end:
	45	+ wx.CallAfter(self.on_end, worker.result)
	46	+
	47	+ def print_msg(self, msg):
	48	+ self.text_ctrl.AppendText(msg)
...	...

gui/manager.py 0 → 100755

View file @44cbbee

	1	+++ a/gui/manager.py
	1	+#!/usr/bin/env python
	2	+# -- coding: utf-8 --
	3	+# generated by wxGlade 0.6.3 on Tue Sep 27 12:15:56 2011
	4	+
	5	+import os
	6	+import wx
	7	+
	8	+import i18n
	9	+_ = i18n.language.ugettext #use ugettext instead of getttext to avoid unicode errors
	10	+
	11	+from dfs.config import Config
	12	+from client import run_upload, run_download, run_download_prim, run_checkout, run_return, run_check_stats
	13	+
	14	+from MainFrame import MainFrame
	15	+import utils
	16	+import const
	17	+from ready import Ready
	18	+from dir_cache import DirCache
	19	+from log_thread import LogThread
	20	+from footprints import FootPrints
	21	+
	22	+class FileManager(wx.App):
	23	+
	24	+ def OnInit(self):
	25	+ wx.InitAllImageHandlers()
	26	+ self.main_frame = MainFrame(None, -1, "")
	27	+ self.SetTopWindow(self.main_frame)
	28	+ self.main_frame.Show()
	29	+
	30	+ # Custom code
	31	+ self.main_frame.set_manager(self)
	32	+
	33	+ self.curr_dir = None
	34	+ self.config = None
	35	+ self.selection = None
	36	+ self.connected = False
	37	+ self.dir_cache = DirCache(const.dir_cache_name, size=5)
	38	+ self.refresh_thread = utils.RefreshThread(self, delay=2)
	39	+ self.refresh_thread.start()
	40	+
	41	+ utils.fill_dir_menu(self.main_frame, self.dir_cache.get())
	42	+ # End custom code
	43	+
	44	+ return 1
	45	+
	46	+ def reconf_directory(self):
	47	+ if not self.curr_dir:
	48	+ return
	49	+ config_path = const.path(self.curr_dir, const.config_name)
	50	+ if utils.create_dir_config( config_path, self.main_frame
	51	+ , config=self.config ):
	52	+ self.config = Config(config_path)
	53	+ self.refresh()
	54	+
	55	+ def set_directory(self, path):
	56	+ config_path = const.path(path, const.config_name)
	57	+ if not os.path.exists(config_path):
	58	+ if not utils.create_dir_config(config_path, self.main_frame):
	59	+ return
	60	+ self.config = Config(config_path)
	61	+ self.ready = Ready(const.path(path, const.ready_name))
	62	+ self.curr_dir = path
	63	+ self.dir_cache.add(path)
	64	+ self.main_frame.SetTitle(path)
	65	+ self.refresh()
	66	+
	67	+ def begin_conn(self):
	68	+ if not self.curr_dir:
	69	+ wx.MessageBox(_('Directory not set'), _('Info'))
	70	+ return False
	71	+ if self.connected:
	72	+ wx.MessageBox(_('Another operation running'), _('Info'))
	73	+ return False
	74	+ self.connected = True
	75	+ return True
	76	+
	77	+ def end_conn(self, op_result):
	78	+ self.connected = False
	79	+
	80	+ def upload(self, files, checkin=False):
	81	+ if not self.begin_conn():
	82	+ return
	83	+
	84	+ def on_end(result):
	85	+ self.end_conn(result)
	86	+ if checkin:
	87	+ return
	88	+ if result == True:
	89	+ prints = self.footprints()
	90	+ for file_name in files:
	91	+ prints.renew(file_name)
	92	+ prints.save()
	93	+
	94	+ logger = LogThread( text_ctrl=self.main_frame.log_ctrl
	95	+ , target=run_upload
	96	+ , args=( self.config["login"]
	97	+ , self.config["passwd"]
	98	+ , const.conn_host
	99	+ , const.conn_port
	100	+ , const.conn_cert
	101	+ , files
	102	+ , self.curr_dir
	103	+ , self.config["extensions"].split()
	104	+ , checkin )
	105	+ , on_end=on_end )
	106	+ logger.start()
	107	+
	108	+ def download(self, n):
	109	+ if not self.begin_conn():
	110	+ return
	111	+ logger = LogThread( text_ctrl=self.main_frame.log_ctrl
	112	+ , target=run_download
	113	+ , args=( self.config["login"]
	114	+ , self.config["passwd"]
	115	+ , const.conn_host
	116	+ , const.conn_port
	117	+ , const.conn_cert
	118	+ , n
	119	+ , self.curr_dir )
	120	+ , on_end=self.end_conn )
	121	+ logger.start()
	122	+
	123	+ def download_prim(self, n):
	124	+ if not self.begin_conn():
	125	+ return
	126	+ logger = LogThread( text_ctrl=self.main_frame.log_ctrl
	127	+ , target=run_download_prim
	128	+ , args=( self.config["login"]
	129	+ , self.config["passwd"]
	130	+ , const.conn_host
	131	+ , const.conn_port
	132	+ , const.conn_cert
	133	+ , n
	134	+ , self.curr_dir )
	135	+ , on_end=self.end_conn )
	136	+ logger.start()
	137	+
	138	+ def checkout(self):
	139	+ if not self.begin_conn():
	140	+ return
	141	+
	142	+ def on_end(result):
	143	+ self.end_conn(result)
	144	+ if result == True:
	145	+ prints = self.footprints()
	146	+ files_before = prints.prints.keys()
	147	+ prints.renew_all()
	148	+ prints.save()
	149	+
	150	+
	151	+ logger = LogThread( text_ctrl=self.main_frame.log_ctrl
	152	+ , target=run_checkout
	153	+ , args=( self.config["login"]
	154	+ , self.config["passwd"]
	155	+ , const.conn_host
	156	+ , const.conn_port
	157	+ , const.conn_cert
	158	+ , self.curr_dir
	159	+ , self.config["extensions"].split() )
	160	+ , on_end=on_end )
	161	+ logger.start()
	162	+
	163	+ def return_files(self, files, reason):
	164	+ if not self.begin_conn():
	165	+ return
	166	+
	167	+ def on_end(result):
	168	+ self.end_conn(result)
	169	+
	170	+ logger = LogThread( text_ctrl=self.main_frame.log_ctrl
	171	+ , target=run_return
	172	+ , args=( self.config["login"]
	173	+ , self.config["passwd"]
	174	+ , const.conn_host
	175	+ , const.conn_port
	176	+ , const.conn_cert
	177	+ , files
	178	+ , reason
	179	+ , self.curr_dir
	180	+ , self.config["extensions"].split()
	181	+ )
	182	+ , on_end=on_end )
	183	+ logger.start()
	184	+
	185	+ def footprints(self):
	186	+ return FootPrints( const.path(self.curr_dir, const.footprints_name)
	187	+ , self.curr_dir
	188	+ , self.config["extensions"].split() )
	189	+
	190	+ def refresh(self):
	191	+ if self.curr_dir is None:
	192	+ return
	193	+
	194	+ ready = self.ready.get_entries()
	195	+ extensions = self.config["extensions"].split()
	196	+
	197	+ footprints = self.footprints()
	198	+ footprints.refresh()
	199	+
	200	+ data = utils.normal_entries(
	201	+ self.curr_dir,
	202	+ extensions,
	203	+ ready,
	204	+ footprints)
	205	+
	206	+ shown = self.main_frame.files_list.shown_data()
	207	+
	208	+ if utils.differ(data, shown):
	209	+ print "Data changed !"
	210	+ self.main_frame.files_list.show(data)
	211	+ footprints.save()
	212	+
	213	+ data = utils.super_entries(
	214	+ self.curr_dir,
	215	+ extensions,
	216	+ ready,
	217	+ footprints)
	218	+
	219	+ shown = self.main_frame.super_files_list.shown_data()
	220	+
	221	+ if utils.differ(data, shown):
	222	+ print "Data changed !"
	223	+ self.main_frame.super_files_list.show(data)
	224	+
	225	+ footprints.save()
	226	+
	227	+ def check_stats(self):
	228	+ if not self.begin_conn():
	229	+ return
	230	+
	231	+ def on_end(result):
	232	+ self.end_conn(result)
	233	+
	234	+ logger = LogThread( text_ctrl=self.main_frame.log_ctrl
	235	+ , target=run_check_stats
	236	+ , args=( self.config["login"]
	237	+ , self.config["passwd"]
	238	+ , const.conn_host
	239	+ , const.conn_port
	240	+ , const.conn_cert )
	241	+ , on_end=on_end )
	242	+ logger.start()
	243	+
	244	+# end of class FileManager
	245	+
	246	+if __name__ == "__main__":
	247	+ manager = FileManager(0)
	248	+ manager.MainLoop()
	249	+ manager.refresh_thread.exitFlag = True
...	...

gui/po/en/LC_MESSAGES/manager.po 0 → 100644

View file @44cbbee

	1	+++ a/gui/po/en/LC_MESSAGES/manager.po
	1	+msgid ""
	2	+msgstr ""
	3	+"Project-Id-Version: manager\n"
	4	+"Report-Msgid-Bugs-To: \n"
	5	+"POT-Creation-Date: 2011-12-14 15:46+0100\n"
	6	+"PO-Revision-Date: \n"
	7	+"Last-Translator: Mateusz Kopeć <m.kopec@ipipan.waw.pl>\n"
	8	+"Language-Team: IPI PAN\n"
	9	+"MIME-Version: 1.0\n"
	10	+"Content-Type: text/plain; charset=UTF-8\n"
	11	+"Content-Transfer-Encoding: 8bit\n"
	12	+"X-Poedit-Language: Polish\n"
	13	+"X-Poedit-Country: POLAND\n"
	14	+"X-Poedit-SourceCharset: utf-8\n"
	15	+
	16	+#: .././MainFrame.py:40
	17	+msgid "Open"
	18	+msgstr ""
	19	+
	20	+#: .././MainFrame.py:40
	21	+msgid "Open new directory"
	22	+msgstr ""
	23	+
	24	+#: .././MainFrame.py:42
	25	+msgid "Reconfigure"
	26	+msgstr ""
	27	+
	28	+#: .././MainFrame.py:42
	29	+msgid "Reconfigure directory"
	30	+msgstr ""
	31	+
	32	+#: .././MainFrame.py:45
	33	+msgid "Directory"
	34	+msgstr ""
	35	+
	36	+#: .././MainFrame.py:47
	37	+msgid "Upload Selected"
	38	+msgstr ""
	39	+
	40	+#: .././MainFrame.py:47
	41	+msgid "Upload annotated or adjudicated files"
	42	+msgstr ""
	43	+
	44	+#: .././MainFrame.py:48
	45	+msgid "Checkin Selected"
	46	+msgstr ""
	47	+
	48	+#: .././MainFrame.py:48
	49	+msgid "Upload annotated or adjudicated; files will be deleted from local disk and you will not be able to modify them again"
	50	+msgstr ""
	51	+
	52	+#: .././MainFrame.py:50
	53	+msgid "Download (annotation)"
	54	+msgstr ""
	55	+
	56	+#: .././MainFrame.py:50
	57	+msgid "Download files for annotation"
	58	+msgstr ""
	59	+
	60	+#: .././MainFrame.py:51
	61	+msgid "Download (adjudication)"
	62	+msgstr ""
	63	+
	64	+#: .././MainFrame.py:51
	65	+msgid "Download files for adjudication"
	66	+msgstr ""
	67	+
	68	+#: .././MainFrame.py:53
	69	+msgid "Checkout"
	70	+msgstr ""
	71	+
	72	+#: .././MainFrame.py:53
	73	+msgid "Checkout files from server to your local, empty directory"
	74	+msgstr ""
	75	+
	76	+#: .././MainFrame.py:54
	77	+msgid "Files"
	78	+msgstr ""
	79	+
	80	+#: .././MainFrame.py:56
	81	+#: .././SortableListCtrl.py:27
	82	+msgid "Ready"
	83	+msgstr ""
	84	+
	85	+#: .././MainFrame.py:56
	86	+msgid "Select items which are marked for upload"
	87	+msgstr ""
	88	+
	89	+#: .././MainFrame.py:57
	90	+#: .././utils.py:45
	91	+#: .././utils.py:94
	92	+msgid "Modified"
	93	+msgstr ""
	94	+
	95	+#: .././MainFrame.py:57
	96	+msgid "Select items which are modified"
	97	+msgstr ""
	98	+
	99	+#: .././MainFrame.py:58
	100	+msgid "Select"
	101	+msgstr ""
	102	+
	103	+#: .././MainFrame.py:106
	104	+msgid "Annotation"
	105	+msgstr ""
	106	+
	107	+#: .././MainFrame.py:107
	108	+msgid "Adjudication"
	109	+msgstr ""
	110	+
	111	+#: .././MainFrame.py:124
	112	+msgid "Choose a directory"
	113	+msgstr ""
	114	+
	115	+#: .././MainFrame.py:137
	116	+msgid "Do you want to upload the following files?"
	117	+msgstr ""
	118	+
	119	+#: .././MainFrame.py:139
	120	+msgid "Question"
	121	+msgstr ""
	122	+
	123	+#: .././MainFrame.py:192
	124	+msgid "Do you want to checkin the following files?"
	125	+msgstr ""
	126	+
	127	+#: .././MainFrame.py:194
	128	+msgid "Files will be deleted from the local directory and you will not be able to read, write or modify them later."
	129	+msgstr ""
	130	+
	131	+#: .././MainFrame.py:210
	132	+#: .././MainFrame.py:221
	133	+msgid "Starting mmax for text"
	134	+msgstr ""
	135	+
	136	+#: .././MainFrame.py:228
	137	+msgid "Error starting mmax:"
	138	+msgstr ""
	139	+
	140	+#: .././utils.py:46
	141	+#: .././utils.py:82
	142	+#: .././utils.py:90
	143	+#: .././utils.py:122
	144	+msgid "Yes"
	145	+msgstr ""
	146	+
	147	+#: .././utils.py:46
	148	+#: .././utils.py:81
	149	+#: .././utils.py:85
	150	+#: .././utils.py:115
	151	+#: .././utils.py:120
	152	+#: .././utils.py:125
	153	+msgid "No"
	154	+msgstr ""
	155	+
	156	+#: .././manager.py:70
	157	+msgid "Directory not set"
	158	+msgstr ""
	159	+
	160	+#: .././manager.py:70
	161	+#: .././manager.py:73
	162	+msgid "Info"
	163	+msgstr ""
	164	+
	165	+#: .././manager.py:73
	166	+msgid "Another operation running"
	167	+msgstr ""
	168	+
	169	+#: .././ConfigDialog.py:21
	170	+msgid "login"
	171	+msgstr ""
	172	+
	173	+#: .././ConfigDialog.py:23
	174	+msgid "password"
	175	+msgstr ""
	176	+
	177	+#: .././ConfigDialog.py:25
	178	+msgid "extensions"
	179	+msgstr ""
	180	+
	181	+#: .././ConfigDialog.py:27
	182	+#: .././DownloadNumberDialog.py:23
	183	+msgid "OK"
	184	+msgstr ""
	185	+
	186	+#: .././ConfigDialog.py:35
	187	+msgid "Directory configuration"
	188	+msgstr ""
	189	+
	190	+#: .././ConfigDialog.py:36
	191	+msgid "Login supplied by system administrator"
	192	+msgstr ""
	193	+
	194	+#: .././ConfigDialog.py:37
	195	+msgid "Password supplied by system administrator"
	196	+msgstr ""
	197	+
	198	+#: .././ConfigDialog.py:38
	199	+msgid "List of space separated file extensions"
	200	+msgstr ""
	201	+
	202	+#: .././DownloadNumberDialog.py:21
	203	+msgid "Download count"
	204	+msgstr ""
	205	+
	206	+#: .././DownloadNumberDialog.py:24
	207	+msgid "Cancel"
	208	+msgstr ""
	209	+
	210	+#: .././DownloadNumberDialog.py:35
	211	+msgid "Download"
	212	+msgstr ""
	213	+
	214	+#: .././DownloadNumberDialog.py:36
	215	+msgid "Number of files to download"
	216	+msgstr ""
	217	+
	218	+#: .././SortableListCtrl.py:25
	219	+msgid "File"
	220	+msgstr ""
	221	+
	222	+#: .././SortableListCtrl.py:26
	223	+msgid "Status"
	224	+msgstr ""
	225	+
	226	+#: .././SortableListCtrl.py:28
	227	+msgid "Stage"
	228	+msgstr ""
	229	+
...	...

gui/po/generate_pot.sh 0 → 100755

View file @44cbbee

	1	+++ a/gui/po/generate_pot.sh
	1	+xgettext --language=Python --keyword=_ --output=manager.pot `find ../. -name "*.py"`
...	...

gui/po/manager.pot 0 → 100644

View file @44cbbee

	1	+++ a/gui/po/manager.pot
	1	+# SOME DESCRIPTIVE TITLE.
	2	+# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
	3	+# This file is distributed under the same license as the PACKAGE package.
	4	+# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
	5	+#
	6	+#, fuzzy
	7	+msgid ""
	8	+msgstr ""
	9	+"Project-Id-Version: PACKAGE VERSION\n"
	10	+"Report-Msgid-Bugs-To: \n"
	11	+"POT-Creation-Date: 2012-06-27 16:25+0200\n"
	12	+"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
	13	+"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
	14	+"Language-Team: LANGUAGE <LL@li.org>\n"
	15	+"Language: \n"
	16	+"MIME-Version: 1.0\n"
	17	+"Content-Type: text/plain; charset=CHARSET\n"
	18	+"Content-Transfer-Encoding: 8bit\n"
	19	+
	20	+#: .././utils.py:46 .././MainFrame.py:62 .././SortableListCtrl.py:130
	21	+msgid "Modified"
	22	+msgstr ""
	23	+
	24	+#: .././utils.py:47 .././SortableListCtrl.py:78 .././SortableListCtrl.py:106
	25	+#: .././SortableListCtrl.py:124 .././SortableListCtrl.py:127
	26	+msgid "Yes"
	27	+msgstr ""
	28	+
	29	+#: .././utils.py:47 .././SortableListCtrl.py:110
	30	+msgid "No"
	31	+msgstr ""
	32	+
	33	+#: .././DownloadNumberDialog.py:21
	34	+msgid "Download count"
	35	+msgstr ""
	36	+
	37	+#: .././DownloadNumberDialog.py:23 .././ReturnReasonDialog.py:27
	38	+#: .././ConfigDialog.py:27
	39	+msgid "OK"
	40	+msgstr ""
	41	+
	42	+#: .././DownloadNumberDialog.py:24 .././ReturnReasonDialog.py:28
	43	+msgid "Cancel"
	44	+msgstr ""
	45	+
	46	+#: .././DownloadNumberDialog.py:35
	47	+msgid "Download"
	48	+msgstr ""
	49	+
	50	+#: .././DownloadNumberDialog.py:36
	51	+msgid "Number of files to download"
	52	+msgstr ""
	53	+
	54	+#: .././ReturnReasonDialog.py:21
	55	+msgid "Why do you want to return the following files?"
	56	+msgstr ""
	57	+
	58	+#: .././ReturnReasonDialog.py:25
	59	+msgid "Reason"
	60	+msgstr ""
	61	+
	62	+#: .././ReturnReasonDialog.py:39
	63	+msgid "Return files"
	64	+msgstr ""
	65	+
	66	+#: .././ConfigDialog.py:21
	67	+msgid "login"
	68	+msgstr ""
	69	+
	70	+#: .././ConfigDialog.py:23
	71	+msgid "password"
	72	+msgstr ""
	73	+
	74	+#: .././ConfigDialog.py:25
	75	+msgid "extensions"
	76	+msgstr ""
	77	+
	78	+#: .././ConfigDialog.py:35
	79	+msgid "Directory configuration"
	80	+msgstr ""
	81	+
	82	+#: .././ConfigDialog.py:36
	83	+msgid "Login supplied by system administrator"
	84	+msgstr ""
	85	+
	86	+#: .././ConfigDialog.py:37
	87	+msgid "Password supplied by system administrator"
	88	+msgstr ""
	89	+
	90	+#: .././ConfigDialog.py:38
	91	+msgid "List of space separated file extensions"
	92	+msgstr ""
	93	+
	94	+#: .././MainFrame.py:43 .././SortableListCtrl.py:85
	95	+msgid "Open"
	96	+msgstr ""
	97	+
	98	+#: .././MainFrame.py:43
	99	+msgid "Open new directory"
	100	+msgstr ""
	101	+
	102	+#: .././MainFrame.py:45
	103	+msgid "Reconfigure"
	104	+msgstr ""
	105	+
	106	+#: .././MainFrame.py:45
	107	+msgid "Reconfigure directory"
	108	+msgstr ""
	109	+
	110	+#: .././MainFrame.py:48
	111	+msgid "Directory"
	112	+msgstr ""
	113	+
	114	+#: .././MainFrame.py:50
	115	+msgid "Upload Selected"
	116	+msgstr ""
	117	+
	118	+#: .././MainFrame.py:50
	119	+msgid "Upload annotated or adjudicated files"
	120	+msgstr ""
	121	+
	122	+#: .././MainFrame.py:51
	123	+msgid "Checkin Selected"
	124	+msgstr ""
	125	+
	126	+#: .././MainFrame.py:51
	127	+msgid ""
	128	+"Upload annotated or adjudicated; files will be deleted from local disk and "
	129	+"you will not be able to modify them again"
	130	+msgstr ""
	131	+
	132	+#: .././MainFrame.py:52
	133	+msgid "Return Selected"
	134	+msgstr ""
	135	+
	136	+#: .././MainFrame.py:52
	137	+msgid ""
	138	+"Return annotated or adjudicated because they are not suitable for annotation "
	139	+"or contain errors"
	140	+msgstr ""
	141	+
	142	+#: .././MainFrame.py:54
	143	+msgid "Download (annotation)"
	144	+msgstr ""
	145	+
	146	+#: .././MainFrame.py:54
	147	+msgid "Download files for annotation"
	148	+msgstr ""
	149	+
	150	+#: .././MainFrame.py:55
	151	+msgid "Download (adjudication)"
	152	+msgstr ""
	153	+
	154	+#: .././MainFrame.py:55
	155	+msgid "Download files for adjudication"
	156	+msgstr ""
	157	+
	158	+#: .././MainFrame.py:57
	159	+msgid "Checkout"
	160	+msgstr ""
	161	+
	162	+#: .././MainFrame.py:57
	163	+msgid "Checkout files from server to your local, empty directory"
	164	+msgstr ""
	165	+
	166	+#: .././MainFrame.py:59
	167	+msgid "Files"
	168	+msgstr ""
	169	+
	170	+#: .././MainFrame.py:61 .././SortableListCtrl.py:49
	171	+msgid "Ready"
	172	+msgstr ""
	173	+
	174	+#: .././MainFrame.py:61
	175	+msgid "Select items which are marked for upload"
	176	+msgstr ""
	177	+
	178	+#: .././MainFrame.py:62
	179	+msgid "Select items which are modified"
	180	+msgstr ""
	181	+
	182	+#: .././MainFrame.py:63
	183	+msgid "Select"
	184	+msgstr ""
	185	+
	186	+#: .././MainFrame.py:66
	187	+msgid "Finished count"
	188	+msgstr ""
	189	+
	190	+#: .././MainFrame.py:66
	191	+msgid "Check number of finished texts"
	192	+msgstr ""
	193	+
	194	+#: .././MainFrame.py:67 .././MainFrame.py:110
	195	+msgid "Annotation"
	196	+msgstr ""
	197	+
	198	+#: .././MainFrame.py:111
	199	+msgid "Adjudication"
	200	+msgstr ""
	201	+
	202	+#: .././MainFrame.py:128
	203	+msgid "Choose a directory"
	204	+msgstr ""
	205	+
	206	+#: .././MainFrame.py:141
	207	+msgid "Do you want to upload the following files?"
	208	+msgstr ""
	209	+
	210	+#: .././MainFrame.py:143 .././MainFrame.py:212
	211	+msgid "Question"
	212	+msgstr ""
	213	+
	214	+#: .././MainFrame.py:209
	215	+msgid "Do you want to checkin the following files?"
	216	+msgstr ""
	217	+
	218	+#: .././MainFrame.py:211
	219	+msgid ""
	220	+"Files will be deleted from the local directory and you will not be able to "
	221	+"read, write or modify them later."
	222	+msgstr ""
	223	+
	224	+#: .././manager.py:69
	225	+msgid "Directory not set"
	226	+msgstr ""
	227	+
	228	+#: .././manager.py:69 .././manager.py:72
	229	+msgid "Info"
	230	+msgstr ""
	231	+
	232	+#: .././manager.py:72
	233	+msgid "Another operation running"
	234	+msgstr ""
	235	+
	236	+#: .././client.py:23
	237	+msgid "Connecting server..."
	238	+msgstr ""
	239	+
	240	+#: .././client.py:35 .././client.py:87 .././client.py:113 .././client.py:155
	241	+#: .././client.py:164 .././client.py:234
	242	+msgid "Sending request..."
	243	+msgstr ""
	244	+
	245	+#: .././client.py:44 .././client.py:242
	246	+msgid "Uploading"
	247	+msgstr ""
	248	+
	249	+#: .././client.py:57 .././client.py:255
	250	+msgid "Upload failed: server doesn't accept this file from you"
	251	+msgstr ""
	252	+
	253	+#: .././client.py:66 .././client.py:82 .././client.py:108 .././client.py:150
	254	+#: .././client.py:161 .././client.py:226 .././client.py:275
	255	+msgid "Done"
	256	+msgstr ""
	257	+
	258	+#: .././client.py:74 .././client.py:79 .././client.py:267 .././client.py:272
	259	+msgid "Deleting"
	260	+msgstr ""
	261	+
	262	+#: .././client.py:91 .././client.py:124
	263	+msgid "No files to download"
	264	+msgstr ""
	265	+
	266	+#: .././client.py:96 .././client.py:129 .././client.py:171 .././client.py:183
	267	+msgid "Downloading"
	268	+msgstr ""
	269	+
	270	+#: .././client.py:118
	271	+msgid "FAILED: you don't have adjudicator privileges"
	272	+msgstr ""
	273	+
	274	+#: .././client.py:158
	275	+msgid "Number of finished files: "
	276	+msgstr ""
	277	+
	278	+#: .././client.py:202
	279	+msgid "Deleting local files..."
	280	+msgstr ""
	281	+
	282	+#: .././client.py:218
	283	+msgid "Error occured:"
	284	+msgstr ""
	285	+
	286	+#: .././client.py:220
	287	+msgid "Saving dowloaded files..."
	288	+msgstr ""
	289	+
	290	+#: .././client.py:299
	291	+msgid "Authentication..."
	292	+msgstr ""
	293	+
	294	+#: .././client.py:304
	295	+msgid "FAILED: incorrect login or password"
	296	+msgstr ""
	297	+
	298	+#: .././client.py:318 .././client.py:339 .././client.py:356 .././client.py:377
	299	+#: .././client.py:398 .././client.py:415
	300	+msgid "FAILED: unable to connect to server"
	301	+msgstr ""
	302	+
	303	+#: .././SortableListCtrl.py:47
	304	+msgid "File"
	305	+msgstr ""
	306	+
	307	+#: .././SortableListCtrl.py:48
	308	+msgid "Status"
	309	+msgstr ""
	310	+
	311	+#: .././SortableListCtrl.py:50
	312	+msgid "Stage"
	313	+msgstr ""
	314	+
	315	+#: .././SortableListCtrl.py:150
	316	+msgid "Starting "
	317	+msgstr ""
	318	+
	319	+#: .././SortableListCtrl.py:150
	320	+msgid " for text"
	321	+msgstr ""
	322	+
	323	+#: .././SortableListCtrl.py:159
	324	+msgid "Error starting "
	325	+msgstr ""
...	...

gui/po/pl/LC_MESSAGES/manager.po 0 → 100644

View file @44cbbee

	1	+++ a/gui/po/pl/LC_MESSAGES/manager.po
	1	+msgid ""
	2	+msgstr ""
	3	+"Project-Id-Version: manager\n"
	4	+"Report-Msgid-Bugs-To: \n"
	5	+"POT-Creation-Date: 2012-06-27 16:25+0200\n"
	6	+"PO-Revision-Date: \n"
	7	+"Last-Translator: Mateusz Kopeć <m.kopec@ipipan.waw.pl>\n"
	8	+"Language-Team: IPI PAN\n"
	9	+"Language: \n"
	10	+"MIME-Version: 1.0\n"
	11	+"Content-Type: text/plain; charset=UTF-8\n"
	12	+"Content-Transfer-Encoding: 8bit\n"
	13	+"X-Poedit-Language: Polish\n"
	14	+"X-Poedit-Country: POLAND\n"
	15	+"X-Poedit-SourceCharset: utf-8\n"
	16	+
	17	+#: .././utils.py:46
	18	+#: .././MainFrame.py:62
	19	+#: .././SortableListCtrl.py:130
	20	+msgid "Modified"
	21	+msgstr "Zmienione"
	22	+
	23	+#: .././utils.py:47
	24	+#: .././SortableListCtrl.py:78
	25	+#: .././SortableListCtrl.py:106
	26	+#: .././SortableListCtrl.py:124
	27	+#: .././SortableListCtrl.py:127
	28	+msgid "Yes"
	29	+msgstr "Tak"
	30	+
	31	+#: .././utils.py:47
	32	+#: .././SortableListCtrl.py:110
	33	+msgid "No"
	34	+msgstr "Nie"
	35	+
	36	+#: .././DownloadNumberDialog.py:21
	37	+msgid "Download count"
	38	+msgstr "Liczba tekstów do pobrania"
	39	+
	40	+#: .././DownloadNumberDialog.py:23
	41	+#: .././ReturnReasonDialog.py:27
	42	+#: .././ConfigDialog.py:27
	43	+msgid "OK"
	44	+msgstr "OK"
	45	+
	46	+#: .././DownloadNumberDialog.py:24
	47	+#: .././ReturnReasonDialog.py:28
	48	+msgid "Cancel"
	49	+msgstr "Anuluj"
	50	+
	51	+#: .././DownloadNumberDialog.py:35
	52	+msgid "Download"
	53	+msgstr "Pobieranie"
	54	+
	55	+#: .././DownloadNumberDialog.py:36
	56	+msgid "Number of files to download"
	57	+msgstr "Liczba nowych tekstów do pobrania z serwera"
	58	+
	59	+#: .././ReturnReasonDialog.py:21
	60	+msgid "Why do you want to return the following files?"
	61	+msgstr "Dlaczego chcesz odesłać poniższe teksty?"
	62	+
	63	+#: .././ReturnReasonDialog.py:25
	64	+msgid "Reason"
	65	+msgstr "Przyczyna"
	66	+
	67	+#: .././ReturnReasonDialog.py:39
	68	+msgid "Return files"
	69	+msgstr "Odeślij teksty"
	70	+
	71	+#: .././ConfigDialog.py:21
	72	+msgid "login"
	73	+msgstr "Login"
	74	+
	75	+#: .././ConfigDialog.py:23
	76	+msgid "password"
	77	+msgstr "Hasło"
	78	+
	79	+#: .././ConfigDialog.py:25
	80	+msgid "extensions"
	81	+msgstr "Rozszerzenia"
	82	+
	83	+#: .././ConfigDialog.py:35
	84	+msgid "Directory configuration"
	85	+msgstr "Konfiguracja katalogu"
	86	+
	87	+#: .././ConfigDialog.py:36
	88	+msgid "Login supplied by system administrator"
	89	+msgstr "Login dostarczony przez administratora systemu"
	90	+
	91	+#: .././ConfigDialog.py:37
	92	+msgid "Password supplied by system administrator"
	93	+msgstr "Hasło dostarczone przez administratora systemu"
	94	+
	95	+#: .././ConfigDialog.py:38
	96	+msgid "List of space separated file extensions"
	97	+msgstr "Lista rozszerzeń plików oddzielona spacjami"
	98	+
	99	+#: .././MainFrame.py:43
	100	+#: .././SortableListCtrl.py:85
	101	+msgid "Open"
	102	+msgstr "Otwórz"
	103	+
	104	+#: .././MainFrame.py:43
	105	+msgid "Open new directory"
	106	+msgstr "Otwórz nowy katalog."
	107	+
	108	+#: .././MainFrame.py:45
	109	+msgid "Reconfigure"
	110	+msgstr "Zmień konfigurację"
	111	+
	112	+#: .././MainFrame.py:45
	113	+msgid "Reconfigure directory"
	114	+msgstr "Zmień konfigurację aktualnego katalogu - hasło i login anotatora oraz rozszerzenia plików."
	115	+
	116	+#: .././MainFrame.py:48
	117	+msgid "Directory"
	118	+msgstr "Katalog tekstów"
	119	+
	120	+#: .././MainFrame.py:50
	121	+msgid "Upload Selected"
	122	+msgstr "Zapisz zaznaczone teksty na serwerze"
	123	+
	124	+#: .././MainFrame.py:50
	125	+msgid "Upload annotated or adjudicated files"
	126	+msgstr "Zapisz zaznaczone teksty na serwerze. Nie zostaną usunięte z Twojego komputera."
	127	+
	128	+#: .././MainFrame.py:51
	129	+msgid "Checkin Selected"
	130	+msgstr "Zakończ pracę nad zaznaczonymi tekstami"
	131	+
	132	+#: .././MainFrame.py:51
	133	+msgid "Upload annotated or adjudicated; files will be deleted from local disk and you will not be able to modify them again"
	134	+msgstr "Zakończ pracę nad zaznaczonymi tekstami. Zostaną one usunięte z Twojego komputera."
	135	+
	136	+#: .././MainFrame.py:52
	137	+msgid "Return Selected"
	138	+msgstr "Odeślij zaznaczone teksty"
	139	+
	140	+#: .././MainFrame.py:52
	141	+msgid "Return annotated or adjudicated because they are not suitable for annotation or contain errors"
	142	+msgstr "Odeślij anotowane lub superanotowane teksty ponieważ nie nadają się do anotacji lub zawierają błędy"
	143	+
	144	+#: .././MainFrame.py:54
	145	+msgid "Download (annotation)"
	146	+msgstr "Pobierz nowe teksty (anotacja)"
	147	+
	148	+#: .././MainFrame.py:54
	149	+msgid "Download files for annotation"
	150	+msgstr "Pobierz nowe teksty do anotacji."
	151	+
	152	+#: .././MainFrame.py:55
	153	+msgid "Download (adjudication)"
	154	+msgstr "Pobierz nowe teksty (superanotacja)"
	155	+
	156	+#: .././MainFrame.py:55
	157	+msgid "Download files for adjudication"
	158	+msgstr "Pobierz nowe teksty do superanotacji."
	159	+
	160	+#: .././MainFrame.py:57
	161	+msgid "Checkout"
	162	+msgstr "Odtwórz stan z serwera lokalnie"
	163	+
	164	+#: .././MainFrame.py:57
	165	+msgid "Checkout files from server to your local, empty directory"
	166	+msgstr "Pobiera wszystkie pliki przypisane do Ciebie z serwera. Nadpisuje lokalne zmiany."
	167	+
	168	+#: .././MainFrame.py:59
	169	+msgid "Files"
	170	+msgstr "Teksty"
	171	+
	172	+#: .././MainFrame.py:61
	173	+#: .././SortableListCtrl.py:49
	174	+msgid "Ready"
	175	+msgstr "Zakończone"
	176	+
	177	+#: .././MainFrame.py:61
	178	+msgid "Select items which are marked for upload"
	179	+msgstr "Zaznacz teksty, które są oznaczone jako \"Zakończone\"."
	180	+
	181	+#: .././MainFrame.py:62
	182	+msgid "Select items which are modified"
	183	+msgstr "Zaznacz teksty, które są oznaczone jako \"Zmienione\"."
	184	+
	185	+#: .././MainFrame.py:63
	186	+msgid "Select"
	187	+msgstr "Zaznacz teksty"
	188	+
	189	+#: .././MainFrame.py:66
	190	+msgid "Finished count"
	191	+msgstr "Liczba zakończonych tekstów"
	192	+
	193	+#: .././MainFrame.py:66
	194	+msgid "Check number of finished texts"
	195	+msgstr "Sprawdź liczbę zakończonych tekstów"
	196	+
	197	+#: .././MainFrame.py:67
	198	+#: .././MainFrame.py:110
	199	+msgid "Annotation"
	200	+msgstr "Anotacja"
	201	+
	202	+#: .././MainFrame.py:111
	203	+msgid "Adjudication"
	204	+msgstr "Superanotacja"
	205	+
	206	+#: .././MainFrame.py:128
	207	+msgid "Choose a directory"
	208	+msgstr "Wybierz katalog"
	209	+
	210	+#: .././MainFrame.py:141
	211	+msgid "Do you want to upload the following files?"
	212	+msgstr "Czy chcesz zapisać na serwerze poniższe teksty?"
	213	+
	214	+#: .././MainFrame.py:143
	215	+#: .././MainFrame.py:212
	216	+msgid "Question"
	217	+msgstr "Pytanie"
	218	+
	219	+#: .././MainFrame.py:209
	220	+msgid "Do you want to checkin the following files?"
	221	+msgstr "Czy chcesz zakończyć pracę nad poniższymi tekstami?"
	222	+
	223	+#: .././MainFrame.py:211
	224	+msgid "Files will be deleted from the local directory and you will not be able to read, write or modify them later."
	225	+msgstr "Teksty zostaną skasowane z tego komputera i nie będziesz mógł/mogła ich odczytać ani edytować kiedykolwiek później."
	226	+
	227	+#: .././manager.py:69
	228	+msgid "Directory not set"
	229	+msgstr "Katalog nie został wybrany"
	230	+
	231	+#: .././manager.py:69
	232	+#: .././manager.py:72
	233	+msgid "Info"
	234	+msgstr "Informacja"
	235	+
	236	+#: .././manager.py:72
	237	+msgid "Another operation running"
	238	+msgstr "Inna operacja w toku"
	239	+
	240	+#: .././client.py:23
	241	+msgid "Connecting server..."
	242	+msgstr "Łączenie z serwerem..."
	243	+
	244	+#: .././client.py:35
	245	+#: .././client.py:87
	246	+#: .././client.py:113
	247	+#: .././client.py:155
	248	+#: .././client.py:164
	249	+#: .././client.py:234
	250	+msgid "Sending request..."
	251	+msgstr "Wysyłanie żądania..."
	252	+
	253	+#: .././client.py:44
	254	+#: .././client.py:242
	255	+msgid "Uploading"
	256	+msgstr "Wysyłanie"
	257	+
	258	+#: .././client.py:57
	259	+#: .././client.py:255
	260	+msgid "Upload failed: server doesn't accept this file from you"
	261	+msgstr "Wysyłanie nie powiodło się: serwer nie akceptuje tego tekstu od Ciebie."
	262	+
	263	+#: .././client.py:66
	264	+#: .././client.py:82
	265	+#: .././client.py:108
	266	+#: .././client.py:150
	267	+#: .././client.py:161
	268	+#: .././client.py:226
	269	+#: .././client.py:275
	270	+msgid "Done"
	271	+msgstr "Gotowe."
	272	+
	273	+#: .././client.py:74
	274	+#: .././client.py:79
	275	+#: .././client.py:267
	276	+#: .././client.py:272
	277	+msgid "Deleting"
	278	+msgstr "Usuwanie"
	279	+
	280	+#: .././client.py:91
	281	+#: .././client.py:124
	282	+msgid "No files to download"
	283	+msgstr "Brak nowych tekstów do pobrania z serwera (lub pobrano już maksymalną ilość)."
	284	+
	285	+#: .././client.py:96
	286	+#: .././client.py:129
	287	+#: .././client.py:171
	288	+#: .././client.py:183
	289	+msgid "Downloading"
	290	+msgstr "Pobieranie"
	291	+
	292	+#: .././client.py:118
	293	+msgid "FAILED: you don't have adjudicator privileges"
	294	+msgstr "Nie powiodło się: nie masz uprawnień superanotatora."
	295	+
	296	+#: .././client.py:158
	297	+msgid "Number of finished files: "
	298	+msgstr "Liczba zakończonych tekstów:"
	299	+
	300	+#: .././client.py:202
	301	+msgid "Deleting local files..."
	302	+msgstr "Usuwanie lokalnych tekstów..."
	303	+
	304	+#: .././client.py:218
	305	+msgid "Error occured:"
	306	+msgstr "Wystąpił błąd:"
	307	+
	308	+#: .././client.py:220
	309	+msgid "Saving dowloaded files..."
	310	+msgstr "Zapisywanie pobranych tekstów..."
	311	+
	312	+#: .././client.py:299
	313	+msgid "Authentication..."
	314	+msgstr "Uwierzytelnianie..."
	315	+
	316	+#: .././client.py:304
	317	+msgid "FAILED: incorrect login or password"
	318	+msgstr "Nie powiodło się: nieprawidłowy login lub hasło."
	319	+
	320	+#: .././client.py:318
	321	+#: .././client.py:339
	322	+#: .././client.py:356
	323	+#: .././client.py:377
	324	+#: .././client.py:398
	325	+#: .././client.py:415
	326	+msgid "FAILED: unable to connect to server"
	327	+msgstr "Nie powiodło się: problem z łącznością z serwerem."
	328	+
	329	+#: .././SortableListCtrl.py:47
	330	+msgid "File"
	331	+msgstr "Tekst"
	332	+
	333	+#: .././SortableListCtrl.py:48
	334	+msgid "Status"
	335	+msgstr "Status"
	336	+
	337	+#: .././SortableListCtrl.py:50
	338	+msgid "Stage"
	339	+msgstr "Etap"
	340	+
	341	+#: .././SortableListCtrl.py:150
	342	+msgid "Starting "
	343	+msgstr "Uruchamianie "
	344	+
	345	+#: .././SortableListCtrl.py:150
	346	+msgid " for text"
	347	+msgstr " dla tekstu "
	348	+
	349	+#: .././SortableListCtrl.py:159
	350	+msgid "Error starting "
	351	+msgstr "Błąd podczas uruchamiania "
	352	+
	353	+#~ msgid "Starting mmax for text"
	354	+#~ msgstr "Uruchamianie mmax dla tekstu"
...	...

gui/ready.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/ready.py
	1	+# -- coding: utf-8 --
	2	+import os
	3	+
	4	+class Ready:
	5	+
	6	+ """Class for ready file manipulation."""
	7	+
	8	+ def __init__(self, ready_path):
	9	+ if not os.path.exists(ready_path):
	10	+ with open(ready_path, "w") as f: pass
	11	+ self.ready_path = ready_path
	12	+
	13	+ def _check_file(self, file_name, is_ready):
	14	+ entries = self.get_entries()
	15	+ if is_ready == True:
	16	+ entries.append(file_name)
	17	+ else:
	18	+ entries.remove(file_name)
	19	+ self.write_entries(entries)
	20	+
	21	+ def remove_file(self, file_name):
	22	+ self._check_file(file_name, False)
	23	+
	24	+ def save_file(self, file_name):
	25	+ self._check_file(file_name, True)
	26	+
	27	+# def remove_super_file(self, file_name):
	28	+# self._check_file(file_name, False, type="s_checkin")
	29	+#
	30	+# def save_super_file(self, file_name):
	31	+# self._check_file(file_name, True, type="s_checkin")
	32	+
	33	+ def get_entries(self):
	34	+ result = []
	35	+ with open(self.ready_path) as inp:
	36	+ result = [x.strip() for x in inp.readlines()]
	37	+ return result
	38	+
	39	+ def write_entries(self, entries):
	40	+ with open(self.ready_path, "w") as out:
	41	+ for entry in entries:
	42	+ print >> out, entry
...	...

gui/utils.py 0 → 100644

View file @44cbbee

	1	+++ a/gui/utils.py
	1	+# -- coding: utf-8 --
	2	+import os
	3	+import re
	4	+import wx
	5	+import threading
	6	+import time
	7	+
	8	+import i18n
	9	+_ = i18n.language.ugettext #use ugettext instead of getttext to avoid unicode errors
	10	+
	11	+import wx.lib.mixins.listctrl as listmix
	12	+
	13	+from ConfigDialog import ConfigDialog
	14	+import const
	15	+
	16	+def match_ext(name, exts):
	17	+ return any(name.endswith(ext) for ext in exts)
	18	+
	19	+def cut_ext(name, exts):
	20	+ return re.sub("(%s)$" % "\|".join(exts), "", name)
	21	+
	22	+def normal_files(work_dir, exts):
	23	+ names = set(cut_ext(name, exts)
	24	+ for name in os.listdir(work_dir)
	25	+ if match_ext(name, exts))
	26	+ return sorted(names)
	27	+
	28	+def super_files(work_dir, exts):
	29	+ names = []
	30	+ for name in os.listdir(work_dir):
	31	+ path = os.path.join(work_dir, name)
	32	+ if os.path.isdir(path):
	33	+ local = set(cut_ext(name, exts)
	34	+ for name in os.listdir(path)
	35	+ if match_ext(name, exts))
	36	+ names.extend(os.path.join(name, tail) for tail in local)
	37	+ return sorted(names)
	38	+
	39	+def all_files(work_dir, exts):
	40	+ return normal_files(work_dir, exts) + super_files(work_dir, exts)
	41	+
	42	+def get_entries(get_names, work_dir, exts, ready, footprints):
	43	+ names = get_names(work_dir, exts)
	44	+ result = []
	45	+ for file_name in names:
	46	+ status = _("Modified") if footprints.modified(file_name) else "--"
	47	+ is_ready = _("Yes") if file_name in ready else _("No")
	48	+ stage = "--"
	49	+ result.append((file_name, status, is_ready, stage))
	50	+ return result
	51	+
	52	+def normal_entries(work_dir, exts, ready, footprints):
	53	+ return get_entries(normal_files, work_dir, exts, ready, footprints)
	54	+
	55	+def super_entries(work_dir, exts, ready, footprints):
	56	+ return get_entries(super_files, work_dir, exts, ready, footprints)
	57	+
	58	+def differ(data1, data2):
	59	+ if len(data1) != len(data2):
	60	+ return True
	61	+ d1={}
	62	+ for i in range(len(data1)):
	63	+ d1[data1[i][0]]=data1[i][1:]
	64	+ for i in range(len(data2)):
	65	+ if not d1.has_key(data2[i][0]):
	66	+ return True
	67	+ if d1[data2[i][0]] != data2[i][1:]:
	68	+ return True
	69	+ return False
	70	+
	71	+def change_ready(ready, file_list, pos):
	72	+ file_name = file_list.GetFilename(pos)
	73	+ if file_list.IsReady(pos):
	74	+ file_list.SetReadyNo(pos)
	75	+ ready.remove_file(file_name)
	76	+ else:
	77	+ file_list.SetReadyYes(pos)
	78	+ ready.save_file(file_name)
	79	+
	80	+def change_ready_prim(ready, super_file_list, pos):
	81	+ def set_no(dirname):
	82	+ for i in range(super_file_list.GetItemCount()):
	83	+ other_name = super_file_list.GetFilename(i)
	84	+ if os.path.dirname(other_name) == dirname:
	85	+ super_file_list.SetReadyNo(i)
	86	+ if other_name in ready.get_entries():
	87	+ ready.remove_file(other_name)
	88	+
	89	+ file_name = super_file_list.GetFilename(pos)
	90	+ if super_file_list.IsReady(pos):
	91	+ super_file_list.SetReadyNo(pos)
	92	+ ready.remove_file(file_name)
	93	+ else:
	94	+ set_no(os.path.dirname(file_name))
	95	+ super_file_list.SetReadyYes(pos)
	96	+ ready.save_file(file_name)
	97	+
	98	+def create_dir_config(path, parent, config=None):
	99	+ dialog = ConfigDialog(parent, -1)
	100	+ if config:
	101	+ dialog.login_ctrl.SetValue(config["login"])
	102	+ dialog.passwd_ctrl.SetValue(config["passwd"])
	103	+ dialog.exts_ctrl.SetValue(config["extensions"])
	104	+ else:
	105	+ dialog.exts_ctrl.SetValue(" ".join(const.default_exts))
	106	+ try:
	107	+ if dialog.ShowModal() == wx.ID_OK:
	108	+ login = dialog.login_ctrl.GetValue()
	109	+ password = dialog.passwd_ctrl.GetValue()
	110	+ file_exts = dialog.exts_ctrl.GetValue()
	111	+ with open(path, "w") as conf:
	112	+ print >> conf, "login =", login
	113	+ print >> conf, "passwd =", password
	114	+ print >> conf, "extensions =", file_exts
	115	+ return True
	116	+ finally:
	117	+ dialog.Destroy()
	118	+ return False
	119	+
	120	+def fill_dir_menu(main_frame, cache):
	121	+ dir_menu = main_frame.dir_menu
	122	+ last_dir_set=False
	123	+ for entry in cache:
	124	+ if not os.path.isdir(entry):
	125	+ continue
	126	+ wx_id = wx.NewId()
	127	+ dir_menu.Append(wx_id, entry, kind=wx.ITEM_NORMAL)
	128	+ main_frame.Bind(wx.EVT_MENU, main_frame.open_directory, id=wx_id)
	129	+ if not last_dir_set:
	130	+ cmd = wx.CommandEvent(wx.EVT_MENU.evtType[0])
	131	+ cmd.SetEventObject(main_frame)
	132	+ cmd.SetId(wx_id)
	133	+ main_frame.GetEventHandler().ProcessEvent(cmd)
	134	+ last_dir_set = True
	135	+
	136	+# Watek odswieza informacje o zawartosci pliku to-checkin.txt
	137	+# (moze sie zmienic za posrednictwem TrEda lub wyedytowany recznie).
	138	+class RefreshThread (threading.Thread):
	139	+ def __init__(self, manager, delay=2):
	140	+ self.exitFlag = False
	141	+ self.delay = delay
	142	+ self.manager = manager
	143	+ threading.Thread.__init__(self)
	144	+
	145	+ def run(self):
	146	+ while self.exitFlag != True:
	147	+ wx.CallAfter(self.manager.refresh)
	148	+ time.sleep(self.delay)
	149	+
	150	+# self.mtime = None # message file 'modified time'
	151	+# while self.exitFlag != True:
	152	+# new_mtime = os.stat(self.ready_path).st_mtime
	153	+# if new_mtime != self.mtime:
	154	+# wx.CallAfter(self.manager.refresh)
	155	+# self.mtime = new_mtime
	156	+# time.sleep(self.delay)
	157	+
...	...

installation.sh 0 → 100644

View file @44cbbee

	1	+++ a/installation.sh
	1	+#wersja instalacji dla lokalnego pythona
	2	+
	3	+#subversion
	4	+wget http://subversion.tigris.org/downloads/subversion-1.6.15.tar.gz
	5	+wget http://subversion.tigris.org/downloads/subversion-deps-1.6.15.tar.gz
	6	+tar -xzf subversion-1.6.15.tar.gz
	7	+tar -xzf subversion-deps-1.6.15.tar.gz
	8	+cd subversion-1.6.15
	9	+./configure --prefix=$HOME/subversion --without-apxs --without-berkeley-db --with-ssl LDFLAGS="-L/lib64"
	10	+make
	11	+make install
	12	+
	13	+#python
	14	+wget -c http://python.org/ftp/python/2.7.2/Python-2.7.2.tar.bz2
	15	+tar -jxvf Python-2.7.2.tar.bz2
	16	+cd Python-2.7.2
	17	+./configure --prefix=$HOME/.local
	18	+make
	19	+make install
	20	+echo 'PATH=$PATH:$HOME/.local/bin' >> ~/.bashrc
	21	+source ~/.bashrc
	22	+
	23	+#setuptools ???????????????????
	24	+wget http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz#md5=7df2a529a074f613b509fb44feefe74e
	25	+tar -xzf setuptools-0.6c11.tar.gz
	26	+cd setuptools-0.6c11
	27	+$HOME/.local/bin/python2.7 setup.py install
	28	+
	29	+#pysvn
	30	+wget http://pysvn.barrys-emacs.org/source_kits/pysvn-1.7.5.tar.gz
	31	+tar -xzf pysvn-1.7.5.tar.gz
	32	+cd pysvn-1.7.5
	33	+cd Source
	34	+$HOME/.local/bin/python2.7 setup.py configure --svn-inc-dir=$HOME/subversion/include/subversion-1/ --svn-lib-dir=$HOME/subversion/lib/
	35	+cd ..
	36	+$HOME/.local/bin/python2.7 setup.py install
	37	+
	38	+
	39	+
	40	+
	41	+
	42	+
	43	+
...	...

scripts/add_files.py 0 → 100644

View file @44cbbee

	1	+++ a/scripts/add_files.py
	1	+#!/usr/bin/env python
	2	+
	3	+import sys
	4	+import os
	5	+import re
	6	+import shutil
	7	+from collections import defaultdict
	8	+from optparse import OptionParser
	9	+
	10	+# Solution with no hard coded path would be welcome
	11	+sys.path.append(os.path.join(os.path.dirname(os.path.abspath(sys.argv[0])), ".."))
	12	+
	13	+from dfs.database import Database
	14	+from dfs.repo import Repo
	15	+from dfs.config import Config
	16	+
	17	+def add_file(wc, db, file_id, src_paths, anno_per_file):
	18	+ try:
	19	+ db.add(file_id)
	20	+ except Exception as ex:
	21	+ print str(ex)
	22	+ return False
	23	+
	24	+ print "Adding file "+file_id+":"
	25	+ added = []
	26	+ for src_path in src_paths:
	27	+ try:
	28	+ wc.add(src_path)
	29	+ print "\t - "+src_path
	30	+ added.append(src_path)
	31	+ except Exception as ex:
	32	+ print "\t error: "+str(ex)
	33	+ for a in added:
	34	+ print "\t\t removing already added "+a
	35	+ for b in wc.remove(added, anno_per_file):
	36	+ print "\t\t\t "+b
	37	+
	38	+ db.remove(file_id)
	39	+ return False
	40	+
	41	+ return True
	42	+
	43	+def match_ext(path, exts):
	44	+ for ext in exts:
	45	+ if path.endswith(ext):
	46	+ return ext
	47	+
	48	+def path_id(path, ext):
	49	+ _, filename = os.path.split(path)
	50	+ return re.sub("%s$" % ext, "", filename)
	51	+
	52	+def group_paths(paths, exts):
	53	+ result = defaultdict(list)
	54	+ for path in paths:
	55	+ ext = match_ext(path, exts)
	56	+ if (ext != None):
	57	+ file_id = path_id(path, ext)
	58	+ result[file_id].append(path)
	59	+ return result
	60	+
	61	+def get_rec_paths(paths):
	62	+ result = []
	63	+ for path in paths:
	64	+ if os.path.isdir(path):
	65	+ for dirname, dirnames, filenames in os.walk(path):
	66	+ for filename in filenames:
	67	+ result.append(os.path.join(dirname, filename))
	68	+ else:
	69	+ result.append(path)
	70	+ return result
	71	+
	72	+if __name__ == "__main__":
	73	+ optparser = OptionParser(usage="""usage: %prog [options] CONFIG FILES""")
	74	+ optparser.add_option("--extensions", dest="exts", default=".mmax,_mentions.xml,_words.xml",
	75	+ help="List of comma-separated file extensions")
	76	+ (options, args) = optparser.parse_args()
	77	+ if len(args) < 2:
	78	+ optparser.print_help()
	79	+ sys.exit(0)
	80	+
	81	+ conf_path = args[0]
	82	+ cfg = Config(conf_path)
	83	+ anno_per_file = int(cfg["anno_per_file"])
	84	+ paths = get_rec_paths(args[1:])
	85	+ files = group_paths(paths, options.exts.split(","))
	86	+ wc = Repo(cfg["svn.repository"], cfg["svn.login"], cfg["svn.passwd"])
	87	+ db = Database(wc.db_path(), anno_per_file)
	88	+
	89	+ success = []
	90	+ fail = []
	91	+ for file_id, paths in files.iteritems():
	92	+ if add_file(wc, db, file_id, paths, anno_per_file):
	93	+ success.append(file_id)
	94	+ else:
	95	+ fail.append(file_id)
	96	+
	97	+ db.save()
	98	+ wc.commit("Added files: "+str(success))
	99	+
	100	+ print ""
	101	+ if len(success) > 0:
	102	+ print "Added files: "+str(success)
	103	+ if len(fail) > 0:
	104	+ print "Failed to add files: "+str(fail)
	105	+
0	106	\ No newline at end of file
...	...

scripts/clusters.py 0 → 100755

View file @44cbbee

	1	+++ a/scripts/clusters.py
	1	+#!/usr/bin/env python
	2	+import re
	3	+import sys
	4	+import os
	5	+import re
	6	+import shutil
	7	+from collections import defaultdict
	8	+from optparse import OptionParser
	9	+
	10	+# Solution with no hard coded path would be welcome
	11	+sys.path.append(os.path.join(os.path.dirname(os.path.abspath(sys.argv[0])), ".."))
	12	+
	13	+from dfs.database import Database
	14	+from dfs.repo import Repo
	15	+from dfs.config import Config
	16	+
	17	+def fill(text):
	18	+ return fill_custom(text, 20, " ")
	19	+
	20	+def fill_custom(text, l, sym):
	21	+ to_add = max(0, l - len(text))
	22	+ spaces = ""
	23	+ for i in range(to_add):
	24	+ spaces = spaces + sym
	25	+ return text + spaces
	26	+
	27	+def parse_span(span, words_list):
	28	+ words = []
	29	+
	30	+ w = span.split(",")
	31	+ for fragment in w:
	32	+ f = fragment.split("..")
	33	+ id1 = words_list.index(f[0])
	34	+ id2 = words_list.index(f[-1])
	35	+ for i in range(id1, id2+1):
	36	+ words.append(words_list[i])
	37	+
	38	+ return words
	39	+
	40	+def get_context(ids, words_list, size):
	41	+ first = -1
	42	+ last = -1
	43	+
	44	+ i = 0
	45	+ for i in range(len(words_list)):
	46	+ w = words_list[i]
	47	+ if w in ids:
	48	+ if first == -1:
	49	+ first = i
	50	+ last = i
	51	+
	52	+ first = max(0, first - size)
	53	+ last = min(len(words_list), last + size)
	54	+
	55	+ return words_list[first:last]
	56	+
	57	+def print_file(path):
	58	+ word_id_2_orth = {}
	59	+ words_list = []
	60	+ w = re.compile("<word.* id=\"(.?)\".>(.?)</word>.")
	61	+ with open(path + "_words.xml", "r") as f:
	62	+ for line in f.readlines():
	63	+ groups = w.findall(line)
	64	+ if len(groups) == 1:
	65	+ group = groups[0]
	66	+ id = group[0]
	67	+ orth = group[1]
	68	+ word_id_2_orth[id] = orth
	69	+ words_list.append(id)
	70	+
	71	+ me = re.compile("<markable.id=\"(.?)\".*")
	72	+ sp = re.compile(".span=\"(.?)\".*")
	73	+ co = re.compile(".comment=\"(.?)\".*")
	74	+ mg = re.compile(".mention_group=\"(.?)\".*")
	75	+
	76	+ mention_id_2_span = {}
	77	+ mention_id_2_comment = {}
	78	+ clusters = {}
	79	+ with open(path + "_mentions.xml", "r") as f:
	80	+ for line in f.readlines():
	81	+ groups1 = me.findall(line)
	82	+ groups2 = sp.findall(line)
	83	+ groups3 = co.findall(line)
	84	+
	85	+ if len(groups1) == 1 and len(groups2) == 1:
	86	+ id = groups1[0]
	87	+ span = groups2[0]
	88	+ mention_id_2_span[id] = parse_span(span, words_list)
	89	+
	90	+ if len(groups3) == 1:
	91	+ mention_id_2_comment[id] = groups3[0]
	92	+
	93	+ clu = mg.findall(line)
	94	+ if len(clu) == 1 and clu[0] != "empty" and clu[0] != "":
	95	+ if clu[0] not in clusters:
	96	+ clusters[clu[0]] = []
	97	+ clusters[clu[0]].append(id)
	98	+
	99	+ all_clusters = {}
	100	+ for list in clusters.values():
	101	+ l = ["["+" ".join(word_id_2_orth[wid] for wid in mention_id_2_span[id])+"]" for id in list]
	102	+ length = len(l)
	103	+ if length not in all_clusters:
	104	+ all_clusters[length] = []
	105	+ all_clusters[length].append(l)
	106	+
	107	+ return all_clusters
	108	+
	109	+def merge_dicts(dict1, dict2):
	110	+ for key, val in dict2.iteritems():
	111	+ if key in dict1:
	112	+ dict1[key].extend(val)
	113	+ else:
	114	+ dict1[key] = val
	115	+
	116	+def print_clusters(db, min_size):
	117	+ all = {}
	118	+ for filename, file in db.file_index.iteritems():
	119	+
	120	+ if db.rejected(file):
	121	+ continue
	122	+
	123	+ sann = file.find("s_ann")
	124	+
	125	+ if sann is not None and db.finished(sann):
	126	+ path = wc.upload_prim_path(filename)
	127	+ clus = print_file(path)
	128	+ merge_dicts(all, clus)
	129	+
	130	+ for key, val in sorted(all.iteritems()):
	131	+ if key >= min_size:
	132	+ print
	133	+ print "#######", "Dlugosc klastra:", key, "#######"
	134	+ for cl in val:
	135	+ print
	136	+ print ", ".join(cl)
	137	+
	138	+if __name__ == "__main__":
	139	+ optparser = OptionParser(usage="""usage: %prog CONFIG MINCLUSTERSIZE""")
	140	+ (options, args) = optparser.parse_args()
	141	+ if len(args) < 2:
	142	+ optparser.print_help()
	143	+ sys.exit(0)
	144	+
	145	+ conf_path = args[0]
	146	+ min_size = int(args[1])
	147	+ cfg = Config(conf_path)
	148	+ wc = Repo(cfg["svn.repository"], cfg["svn.login"], cfg["svn.passwd"])
	149	+ db = Database(wc.db_path(), int(cfg["anno_per_file"]))
	150	+
	151	+ print_clusters(db, min_size)
	152	+
...	...

scripts/db_averages_stats.py 0 → 100755

View file @44cbbee

	1	+++ a/scripts/db_averages_stats.py
	1	+#!/usr/bin/env python
	2	+import re
	3	+import sys
	4	+import os
	5	+import re
	6	+import shutil
	7	+from collections import defaultdict
	8	+from optparse import OptionParser
	9	+
	10	+# Solution with no hard coded path would be welcome
	11	+sys.path.append(os.path.join(os.path.dirname(os.path.abspath(sys.argv[0])), ".."))
	12	+
	13	+from dfs.database import Database
	14	+from dfs.repo import Repo
	15	+from dfs.config import Config
	16	+
	17	+def fill(text):
	18	+ return fill_custom(text, 20, " ")
	19	+
	20	+def fill_custom(text, l, sym):
	21	+ text = text.decode("utf-8")
	22	+ to_add = max(0, l - len(text))
	23	+ spaces = ""
	24	+ for i in range(to_add):
	25	+ spaces = spaces + sym
	26	+ return (text + spaces).encode("utf-8")
	27	+
	28	+def get_span_size(span, words):
	29	+ w = span.split(",")
	30	+ size = 0
	31	+ for fragment in w:
	32	+ f = fragment.split("..")
	33	+ id1 = words.index(f[0])
	34	+ id2 = words.index(f[-1])
	35	+ size = size + id2 - id1 + 1
	36	+
	37	+ return size
	38	+
	39	+def count_file_stats(path):
	40	+ words = []
	41	+ id = re.compile("<word.* id=\"(.?)\".")
	42	+ with open(path + "_words.xml", "r") as f:
	43	+ for line in f.readlines():
	44	+ groups = id.findall(line)
	45	+ if len(groups) == 1:
	46	+ ident = groups[0]
	47	+ words.append(ident)
	48	+
	49	+ sp = re.compile("<markable.span=\"(.?)\".*")
	50	+ mg = re.compile(".mention_group=\"(.?)\".*")
	51	+ ni = re.compile(".near_identity=\"(.?)\".*")
	52	+
	53	+ sets = {}
	54	+ near_id = 0
	55	+ mentions = 0
	56	+ mention_sizes = {}
	57	+ with open(path + "_mentions.xml", "r") as f:
	58	+ for line in f.readlines():
	59	+ groups = sp.findall(line)
	60	+ if len(groups) == 1:
	61	+ mentions = mentions + 1
	62	+ span = groups[0]
	63	+
	64	+ mention_size = get_span_size(span, words)
	65	+ if mention_size in mention_sizes:
	66	+ mention_sizes[mention_size] = mention_sizes[mention_size] + 1
	67	+ else:
	68	+ mention_sizes[mention_size] = 1
	69	+
	70	+ group = mg.findall(line)[0]
	71	+ near = ni.findall(line)[0]
	72	+
	73	+ if near != "empty" and near != "":
	74	+ near_id = near_id + 1
	75	+
	76	+ if group != "empty" and group != "":
	77	+ if group in sets:
	78	+ sets[group] = sets[group] + 1
	79	+ else:
	80	+ sets[group] = 1
	81	+
	82	+ mg_sizes = {}
	83	+ for key, val in sets.iteritems():
	84	+ if val in mg_sizes:
	85	+ mg_sizes[val] = mg_sizes[val] + 1
	86	+ else:
	87	+ mg_sizes[val] = 1
	88	+
	89	+ return len(words), mention_sizes, mg_sizes, near_id
	90	+
	91	+def merge_dicts(dict1, dict2):
	92	+
	93	+ for key, val in dict2.iteritems():
	94	+ if key in dict1:
	95	+ dict1[key] = dict1[key] + val
	96	+ else:
	97	+ dict1[key] = val
	98	+
	99	+def get_text_types(db, used_path, mapping_path):
	100	+ names = []
	101	+ for filename, file in db.file_index.iteritems():
	102	+ if db.rejected(file):
	103	+ continue
	104	+ names.append(filename)
	105	+
	106	+ id2nrs = {}
	107	+ with open(used_path, "r") as f:
	108	+ for line in f.readlines():
	109	+ spl = line.split(";")
	110	+ id = spl[1]
	111	+ nr = spl[0]
	112	+ if id not in id2nrs:
	113	+ id2nrs[id] = []
	114	+ id2nrs[id].append(nr)
	115	+
	116	+ types = {}
	117	+ with open(mapping_path, "r") as f:
	118	+ for line in f.readlines():
	119	+ spl = line.strip().split(";")
	120	+ id = spl[2]
	121	+ if id in id2nrs.keys():
	122	+ for nr in id2nrs[id]:
	123	+ types[nr] = spl[0].decode("utf-8")
	124	+
	125	+ return types
	126	+
	127	+def print_stats(db, ann, types):
	128	+
	129	+ gms = {}
	130	+ gmgs = {}
	131	+ wbalance = {}
	132	+ tbalance = {}
	133	+ nibalance = {}
	134	+
	135	+ for filename, file in db.file_index.iteritems():
	136	+
	137	+ type = types[filename]
	138	+ idx = 0
	139	+
	140	+ if ann == "rejected":
	141	+ if db.rejected(file):
	142	+ path = os.path.join(wc.new_path(), filename)
	143	+ wcnt, ms, mgs, nidcnt = count_file_stats(path)
	144	+
	145	+ merge_dicts(wbalance, {type : wcnt})
	146	+ merge_dicts(tbalance, {type : 1})
	147	+ merge_dicts(nibalance, {type : nidcnt})
	148	+
	149	+ if type not in gms:
	150	+ gms[type] = {}
	151	+ if type not in gmgs:
	152	+ gmgs[type] = {}
	153	+
	154	+ merge_dicts(gms[type], ms)
	155	+ merge_dicts(gmgs[type], mgs)
	156	+
	157	+ else:
	158	+ if db.rejected(file):
	159	+ continue
	160	+
	161	+ for annotation in file.findall(ann):
	162	+ if db.finished(annotation):
	163	+ if ann == "ann":
	164	+ path = wc.upload_path(filename, idx)
	165	+ else:
	166	+ path = wc.upload_prim_path(filename)
	167	+
	168	+ wcnt, ms, mgs, nidcnt = count_file_stats(path)
	169	+
	170	+ merge_dicts(wbalance, {type : wcnt})
	171	+ merge_dicts(tbalance, {type : 1})
	172	+ merge_dicts(nibalance, {type : nidcnt})
	173	+
	174	+ if type not in gms:
	175	+ gms[type] = {}
	176	+ if type not in gmgs:
	177	+ gmgs[type] = {}
	178	+
	179	+ merge_dicts(gms[type], ms)
	180	+ merge_dicts(gmgs[type], mgs)
	181	+
	182	+ idx = idx + 1
	183	+
	184	+
	185	+ gwcnt = reduce((lambda l, a : l + a), wbalance.values(), 0)
	186	+ tc = reduce((lambda l, a : l + a), tbalance.values(), 0)
	187	+ print
	188	+ print "######## Statystyki ilosciowe"
	189	+ print fill_custom("Typ tekstow", 60, " "), fill("Liczba tekstow"), fill("Liczba slow"), fill("Procent slow")
	190	+ print fill_custom("", 120, "-")
	191	+ for type in sorted(set(wbalance.keys()) \| set(tbalance.keys())):
	192	+ percent = round(1.0 * wbalance[type] / gwcnt * 100, 2)
	193	+ print fill_custom(type.encode("utf-8"), 60, " "), fill(str(tbalance[type])), fill(str(wbalance[type])), fill(str(percent) + "%")
	194	+ print fill_custom("", 120, "-")
	195	+ print fill_custom("dowolny", 60, " "), fill(str(tc)), fill(str(gwcnt)), fill("100.0%")
	196	+
	197	+ if ann != "rejected":
	198	+ print
	199	+ print "######## Statystyki wystapien"
	200	+ print fill_custom("Typ tekstow", 60, " "), fill("Wystapien/tekst"), fill("Dl. wystapienia")
	201	+ print fill_custom("", 100, "-")
	202	+ tmc = 0
	203	+ tsc = 0
	204	+ for type in sorted(set(gms.keys())):
	205	+ mention_count = reduce((lambda l, a : l + a), gms[type].values(), 0)
	206	+ segment_count = reduce((lambda a, (k, v) : k * v + a), gms[type].iteritems(), 0)
	207	+ avg_ms = round(1.0 * segment_count / mention_count, 2)
	208	+ avg_mpt = round(1.0 * mention_count / tbalance[type], 2)
	209	+ print fill_custom(type.encode("utf-8"), 60, " "), fill(str(avg_mpt)), fill(str(avg_ms))
	210	+ tmc = tmc + mention_count
	211	+ tsc = tsc + segment_count
	212	+ print fill_custom("", 100, "-")
	213	+
	214	+ if tmc == 0:
	215	+ tavg_ms = 0
	216	+ else:
	217	+ tavg_ms = round(1.0 * tsc / tmc, 2)
	218	+
	219	+ if tc == 0:
	220	+ tavg_mpt = 0
	221	+ else:
	222	+ tavg_mpt = round(1.0 * tmc / tc, 2)
	223	+
	224	+ print fill_custom("dowolny", 60, " "), fill(str(tavg_mpt)), fill(str(tavg_ms))
	225	+
	226	+ print
	227	+ print "######## Statystyki klastrow i linkow"
	228	+ print fill_custom("Typ tekstow", 60, " "), fill("Klastrow/tekst"), fill("Wielkosc klastra"), fill("Linkow/tekst")
	229	+ print fill_custom("", 120, "-")
	230	+ tmc = 0
	231	+ tsc = 0
	232	+ tnic = 0
	233	+ for type in sorted(set(gmgs.keys())):
	234	+ group_count = reduce((lambda l, a : l + a), gmgs[type].values(), 0)
	235	+ mention_count = reduce((lambda a, (k, v) : k * v + a), gmgs[type].iteritems(), 0)
	236	+ avg_ms = round(1.0 * mention_count / group_count, 2)
	237	+ avg_mpt = round(1.0 * group_count / tbalance[type], 2)
	238	+ avg_ni = round(1.0 * nibalance[type] / tbalance[type], 2)
	239	+ print fill_custom(type.encode("utf-8"), 60, " "), fill(str(avg_mpt)), fill(str(avg_ms)), fill(str(avg_ni))
	240	+ tmc = tmc + group_count
	241	+ tsc = tsc + mention_count
	242	+ tnic = tnic + nibalance[type]
	243	+ print fill_custom("", 120, "-")
	244	+
	245	+ if tmc == 0:
	246	+ tavg_ms = 0
	247	+ else:
	248	+ tavg_ms = round(1.0 * tsc / tmc, 2)
	249	+
	250	+ if tc == 0:
	251	+ tavg_mpt = 0
	252	+ tavg_ni = 0
	253	+ else:
	254	+ tavg_mpt = round(1.0 * tmc / tc, 2)
	255	+ tavg_ni = round(1.0 * tnic / tc, 2)
	256	+
	257	+ print fill_custom("dowolny", 60, " "), fill(str(tavg_mpt)), fill(str(tavg_ms)), fill(str(tavg_ni))
	258	+
	259	+
	260	+if __name__ == "__main__":
	261	+ optparser = OptionParser(usage="""usage: %prog CONFIG USED_LIST MAPPING""")
	262	+ (options, args) = optparser.parse_args()
	263	+ if len(args) < 3:
	264	+ optparser.print_help()
	265	+ sys.exit(0)
	266	+
	267	+ conf_path = args[0]
	268	+ used_path = args[1]
	269	+ mapping_path = args[2]
	270	+ cfg = Config(conf_path)
	271	+ wc = Repo(cfg["svn.repository"], cfg["svn.login"], cfg["svn.passwd"])
	272	+ db = Database(wc.db_path(), int(cfg["anno_per_file"]))
	273	+
	274	+ types = get_text_types(db, used_path, mapping_path)
	275	+
	276	+ print "################ Statystyki tekstow anotowanych #############################"
	277	+ print_stats(db, "ann", types)
	278	+ print
	279	+ print "################ Statystyki tekstow superanotowanych ########################"
	280	+ print_stats(db, "s_ann", types)
	281	+ print
	282	+# print "################ Statystyki tekstow odrzuconych #############################"
	283	+# print_stats(db, "rejected", types)
...	...

scripts/db_detailed_stats.py 0 → 100755

View file @44cbbee

	1	+++ a/scripts/db_detailed_stats.py
	1	+# -- coding: utf-8 --
	2	+
	3	+#!/usr/bin/env python
	4	+import re
	5	+import sys
	6	+import os
	7	+import re
	8	+import shutil
	9	+from collections import defaultdict
	10	+from optparse import OptionParser
	11	+
	12	+# Solution with no hard coded path would be welcome
	13	+sys.path.append(os.path.join(os.path.dirname(os.path.abspath(sys.argv[0])), ".."))
	14	+
	15	+from dfs.database import Database
	16	+from dfs.repo import Repo
	17	+from dfs.config import Config
	18	+
	19	+def get_desired():
	20	+ result = {}
	21	+ result[u"Dzienniki"] = 25.5
	22	+ result[u"Pozostałe periodyki"] = 23.5
	23	+ result[u"Książki publicystyczne"] = 1.0
	24	+ result[u"Literatura piękna"] = 16.0
	25	+ result[u"Literatura faktu"] = 5.5
	26	+ result[u"Typ informacyjno-poradnikowy"] = 5.5
	27	+ result[u"Typ naukowo-dydaktyczny"] = 2.0
	28	+ result[u"Internetowe interaktywne (blogi, fora, usenet)"] = 3.5
	29	+ result[u"Internetowe nieinteraktywne (statyczne strony, Wikipedia)"] = 3.5
	30	+ result[u"Quasi-mówione (protokoły sesji parlamentu)"] = 2.5
	31	+ result[u"Mówione medialne"] = 2.5
	32	+ result[u"Mówione konwersacyjne"] = 5.0
	33	+ result[u"Inne teksty pisane"] = 3.0
	34	+ result[u"Książka niebeletrystyczna nieklasyfikowana"] = 1.0
	35	+
	36	+ return result
	37	+
	38	+def fill(text):
	39	+ return fill_custom(text, 20, " ")
	40	+
	41	+def fill_custom(text, l, sym):
	42	+ text = text.decode("utf-8")
	43	+ to_add = max(0, l - len(text))
	44	+ spaces = ""
	45	+ for i in range(to_add):
	46	+ spaces = spaces + sym
	47	+ return (text + spaces).encode("utf-8")
	48	+
	49	+def get_span_size(span, words):
	50	+ w = span.split(",")
	51	+ size = 0
	52	+ for fragment in w:
	53	+ f = fragment.split("..")
	54	+ id1 = words.index(f[0])
	55	+ id2 = words.index(f[-1])
	56	+ size = size + id2 - id1 + 1
	57	+
	58	+ return size
	59	+
	60	+def count_file_stats(path):
	61	+ words = []
	62	+ id = re.compile("<word.* id=\"(.?)\".")
	63	+ with open(path + "_words.xml", "r") as f:
	64	+ for line in f.readlines():
	65	+ groups = id.findall(line)
	66	+ if len(groups) == 1:
	67	+ ident = groups[0]
	68	+ words.append(ident)
	69	+
	70	+ sp = re.compile("<markable.span=\"(.?)\".*")
	71	+ mg = re.compile(".mention_group=\"(.?)\".*")
	72	+ ni = re.compile(".near_identity=\"(.?)\".*")
	73	+
	74	+ sets = {}
	75	+ near_id = 0
	76	+ mentions = 0
	77	+ mention_sizes = {}
	78	+ with open(path + "_mentions.xml", "r") as f:
	79	+ for line in f.readlines():
	80	+ groups = sp.findall(line)
	81	+ if len(groups) == 1:
	82	+ mentions = mentions + 1
	83	+ span = groups[0]
	84	+
	85	+ mention_size = get_span_size(span, words)
	86	+ if mention_size in mention_sizes:
	87	+ mention_sizes[mention_size] = mention_sizes[mention_size] + 1
	88	+ else:
	89	+ mention_sizes[mention_size] = 1
	90	+
	91	+ group = mg.findall(line)[0]
	92	+ near = ni.findall(line)[0]
	93	+
	94	+ if near != "empty" and near != "":
	95	+ near_id = near_id + 1
	96	+
	97	+ if group != "empty" and group != "":
	98	+ if group in sets:
	99	+ sets[group] = sets[group] + 1
	100	+ else:
	101	+ sets[group] = 1
	102	+
	103	+ mg_sizes = {}
	104	+ for key, val in sets.iteritems():
	105	+ if val in mg_sizes:
	106	+ mg_sizes[val] = mg_sizes[val] + 1
	107	+ else:
	108	+ mg_sizes[val] = 1
	109	+
	110	+ return len(words), mention_sizes, mg_sizes, near_id
	111	+
	112	+def merge_dicts(dict1, dict2):
	113	+
	114	+ for key, val in dict2.iteritems():
	115	+ if key in dict1:
	116	+ dict1[key] = dict1[key] + val
	117	+ else:
	118	+ dict1[key] = val
	119	+
	120	+def get_text_types(db, used_path, mapping_path):
	121	+ names = []
	122	+ for filename, file in db.file_index.iteritems():
	123	+ if db.rejected(file):
	124	+ continue
	125	+ names.append(filename)
	126	+
	127	+ id2nrs = {}
	128	+ with open(used_path, "r") as f:
	129	+ for line in f.readlines():
	130	+ spl = line.split(";")
	131	+ id = spl[1]
	132	+ nr = spl[0]
	133	+ if id not in id2nrs:
	134	+ id2nrs[id] = []
	135	+ id2nrs[id].append(nr)
	136	+
	137	+ types = {}
	138	+ with open(mapping_path, "r") as f:
	139	+ for line in f.readlines():
	140	+ spl = line.strip().split(";")
	141	+ id = spl[2]
	142	+ if id in id2nrs.keys():
	143	+ for nr in id2nrs[id]:
	144	+ types[nr] = spl[0].decode("utf-8")
	145	+
	146	+ return types
	147	+
	148	+def print_stats(db, ann, types):
	149	+ gnidcnt = 0
	150	+ gwcnt = 0
	151	+ gms = {}
	152	+ gmgs = {}
	153	+ wbalance = {}
	154	+ tbalance = {}
	155	+
	156	+ tc = 0
	157	+
	158	+ for filename, file in db.file_index.iteritems():
	159	+
	160	+ type = types[filename]
	161	+ idx = 0
	162	+
	163	+ if ann == "rejected":
	164	+ if db.rejected(file):
	165	+ path = os.path.join(wc.new_path(), filename)
	166	+ wcnt, ms, mgs, nidcnt = count_file_stats(path)
	167	+
	168	+ merge_dicts(wbalance, {type : wcnt})
	169	+ merge_dicts(tbalance, {type : 1})
	170	+
	171	+ gwcnt = gwcnt + wcnt
	172	+ gnidcnt = gnidcnt + nidcnt
	173	+ merge_dicts(gms, ms)
	174	+ merge_dicts(gmgs, mgs)
	175	+
	176	+ tc = tc + 1
	177	+
	178	+ else:
	179	+ if db.rejected(file):
	180	+ continue
	181	+
	182	+ for annotation in file.findall(ann):
	183	+ if db.finished(annotation):
	184	+ if ann == "ann":
	185	+ path = wc.upload_path(filename, idx)
	186	+ else:
	187	+ path = wc.upload_prim_path(filename)
	188	+
	189	+ wcnt, ms, mgs, nidcnt = count_file_stats(path)
	190	+
	191	+ merge_dicts(wbalance, {type : wcnt})
	192	+ merge_dicts(tbalance, {type : 1})
	193	+
	194	+ gwcnt = gwcnt + wcnt
	195	+ gnidcnt = gnidcnt + nidcnt
	196	+ merge_dicts(gms, ms)
	197	+ merge_dicts(gmgs, mgs)
	198	+
	199	+ tc = tc + 1
	200	+
	201	+ idx = idx + 1
	202	+
	203	+ desired = get_desired()
	204	+ print
	205	+ print fill_custom("Typ tekstow", 60, " "), fill("Liczba tekstow"), fill("Liczba slow"), fill("Procent slow"), fill("Docelowy procent")
	206	+ print fill_custom("", 150, "-")
	207	+ for type in sorted(set(wbalance.keys()) \| set(tbalance.keys())):
	208	+ percent = round(1.0 * wbalance[type] / gwcnt * 100, 2)
	209	+ print fill_custom(type.encode("utf-8"), 60, " "), fill(str(tbalance[type])), fill(str(wbalance[type])), fill(str(percent) + "%"), fill(str(desired[type])+"%")
	210	+ print fill_custom("", 150, "-")
	211	+ print fill_custom("dowolny", 60, " "), fill(str(tc)), fill(str(gwcnt)), fill("100.0%"), fill("100.0%")
	212	+
	213	+ print
	214	+
	215	+ if ann != "rejected":
	216	+ print "Wielkosci wystapien:"
	217	+ suma = 0
	218	+ for key, val in sorted(gms.iteritems()):
	219	+ print " ", val, "wystapien o wielkosci", key
	220	+ suma = suma + val
	221	+ print " ", "--------------------------"
	222	+ print " ", suma, "wystapien o dowolnej wielkosci"
	223	+
	224	+ print
	225	+
	226	+ print "Wielkosci klastrow:"
	227	+ suma = 0
	228	+ for key, val in sorted(gmgs.iteritems()):
	229	+ print " ", val, "klastrow o wielkosci", key
	230	+ suma = suma + val
	231	+ print " ", "--------------------------"
	232	+ print " ", suma, "klastrow o dowolnej wielkosci"
	233	+
	234	+ print
	235	+
	236	+ print "Liczba linkow:", gnidcnt
	237	+
	238	+
	239	+if __name__ == "__main__":
	240	+ optparser = OptionParser(usage="""usage: %prog CONFIG USED_LIST MAPPING""")
	241	+ (options, args) = optparser.parse_args()
	242	+ if len(args) < 3:
	243	+ optparser.print_help()
	244	+ sys.exit(0)
	245	+
	246	+ conf_path = args[0]
	247	+ used_path = args[1]
	248	+ mapping_path = args[2]
	249	+ cfg = Config(conf_path)
	250	+ wc = Repo(cfg["svn.repository"], cfg["svn.login"], cfg["svn.passwd"])
	251	+ db = Database(wc.db_path(), int(cfg["anno_per_file"]))
	252	+
	253	+ types = get_text_types(db, used_path, mapping_path)
	254	+
	255	+ print "################ Statystyki anotacji #########################"
	256	+ print_stats(db, "ann", types)
	257	+ print
	258	+ print "################ Statystyki superanotacji ####################"
	259	+ print_stats(db, "s_ann", types)
	260	+ print
	261	+# print "################ rejected files stats####################"
	262	+# print_stats(db, "rejected", types)
...	...

scripts/db_stats.py 0 → 100644

View file @44cbbee

	1	+++ a/scripts/db_stats.py
	1	+#!/usr/bin/env python
	2	+
	3	+import sys
	4	+import os
	5	+import re
	6	+import shutil
	7	+from collections import defaultdict
	8	+from optparse import OptionParser
	9	+
	10	+# Solution with no hard coded path would be welcome
	11	+sys.path.append(os.path.join(os.path.dirname(os.path.abspath(sys.argv[0])), ".."))
	12	+
	13	+from dfs.database import Database
	14	+from dfs.repo import Repo
	15	+from dfs.config import Config
	16	+
	17	+def fill(text):
	18	+ return fill_custom(text, 20, " ")
	19	+
	20	+def fill_custom(text, l, sym):
	21	+ to_add = max(0, l - len(text))
	22	+ spaces = ""
	23	+ for i in range(to_add):
	24	+ spaces = spaces + sym
	25	+ return text + spaces
	26	+
	27	+def count_rejected(db):
	28	+ rejected = 0
	29	+ for filename, file in db.file_index.iteritems():
	30	+ if db.rejected(file):
	31	+ rejected = rejected + 1
	32	+ return rejected
	33	+
	34	+def count_anno_stats(db, all, anno_per_file):
	35	+ finished = {}
	36	+ current = {}
	37	+ returned = {}
	38	+ returned_fixed = {}
	39	+ finished_by_all = 0
	40	+ fetched_by_all = 0
	41	+
	42	+ for filename, file in db.file_index.iteritems():
	43	+ if db.rejected(file):
	44	+ continue
	45	+
	46	+ fin = True
	47	+ for annotation in file.findall("ann"):
	48	+ owner = annotation.find("annName").text
	49	+
	50	+ if not owner in finished:
	51	+ finished[owner] = 0
	52	+ if not owner in current:
	53	+ current[owner] = 0
	54	+ if not owner in returned:
	55	+ returned[owner] = 0
	56	+ if not owner in returned_fixed:
	57	+ returned_fixed[owner] = 0
	58	+
	59	+ if db.owned(annotation):
	60	+ current[owner] = current[owner] + 1
	61	+ elif db.returned(annotation):
	62	+ returned[owner] = returned[owner] + 1
	63	+ if db.fixed(annotation):
	64	+ returned_fixed[owner] = returned_fixed[owner] + 1
	65	+ elif db.finished(annotation):
	66	+ finished[owner] = finished[owner] + 1
	67	+ else:
	68	+ print "Strange state! ", annotation.text
	69	+
	70	+ if not db.finished(annotation):
	71	+ fin = False
	72	+
	73	+ if len(file.findall("ann")) == anno_per_file:
	74	+ if fin:
	75	+ finished_by_all = finished_by_all + 1
	76	+ else:
	77	+ fetched_by_all = fetched_by_all + 1
	78	+
	79	+
	80	+ print fill("Login"), fill("Akt. pobr."), fill("Zwrocone (Napr.)"), fill("Zakonczone"), fill("Do pobrania")
	81	+ print fill_custom("", 100, "-")
	82	+ for user in set(current.keys()) \| set(finished.keys()) \| set(returned_fixed.keys()) \| set(returned.keys()):
	83	+ print fill(user), fill(str(current[user])), fill(str(returned[user]) + " (" + str(returned_fixed[user]) + ")"), fill(str(finished[user])),
	84	+ priority, normal = db.for_annotation(user)
	85	+ print fill(str(len(normal)))
	86	+
	87	+ print fill_custom("", 100, "-")
	88	+ all_curr = sum([i for i in current.values()])
	89	+ all_ret = sum([i for i in returned.values()])
	90	+ all_ret_fixed = sum([i for i in returned_fixed.values()])
	91	+ all_fin = sum([i for i in finished.values()])
	92	+ print fill("Suma"), fill(str(all_curr)), fill(str(all_ret) + " (" + str(all_ret_fixed) + ")"), fill(str(all_fin)), fill(str(all - all_curr - all_fin - all_ret))
	93	+ print ""
	94	+ print "Pobranych (nie zakonczonych) jednoczesnie przez", anno_per_file, "tekstow:", fetched_by_all
	95	+ print "Zakonczonych jednoczesnie przez", anno_per_file, "tekstow:", finished_by_all
	96	+ print "Pozostalo do pobrania naprawionych:", all_ret_fixed
	97	+ print "Pozostalo do obejrzenia zwroconych:", all_ret - all_ret_fixed
	98	+ print
	99	+
	100	+ return finished_by_all
	101	+
	102	+
	103	+def count_super_stats(db, all, anno_per_file):
	104	+ finished = {}
	105	+ current = {}
	106	+ returned = {}
	107	+ returned_fixed = {}
	108	+
	109	+ for filename, file in db.file_index.iteritems():
	110	+ if db.rejected(file):
	111	+ continue
	112	+
	113	+ for annotation in file.findall("s_ann"):
	114	+ owner = annotation.find("annName").text
	115	+
	116	+ if not owner in finished:
	117	+ finished[owner] = 0
	118	+ if not owner in current:
	119	+ current[owner] = 0
	120	+ if not owner in returned:
	121	+ returned[owner] = 0
	122	+ if not owner in returned_fixed:
	123	+ returned_fixed[owner] = 0
	124	+
	125	+ if db.owned(annotation):
	126	+ current[owner] = current[owner] + 1
	127	+ elif db.returned(annotation):
	128	+ returned[owner] = returned[owner] + 1
	129	+ if db.fixed(annotation):
	130	+ returned_fixed[owner] = returned_fixed[owner] + 1
	131	+ elif db.finished(annotation):
	132	+ finished[owner] = finished[owner] + 1
	133	+ else:
	134	+ print "Strange state! ", annotation.text
	135	+
	136	+ print fill("Login"), fill("Akt. pobr."), fill("Zwrocone (Napr.)"), fill("Zakonczone"), fill("Do pobrania")
	137	+ print fill_custom("", 100, "-")
	138	+ for user in set(current.keys()) \| set(finished.keys()) \| set(returned_fixed.keys()) \| set(returned.keys()):
	139	+ print fill(user), fill(str(current[user])), fill(str(returned[user]) + " (" + str(returned_fixed[user]) + ")"), fill(str(finished[user])),
	140	+ priority, normal = db.for_adjudication(user)
	141	+ print fill(str(len(normal)))
	142	+
	143	+ print fill_custom("", 100, "-")
	144	+ all_curr = sum([i for i in current.values()])
	145	+ all_ret = sum([i for i in returned.values()])
	146	+ all_ret_fixed = sum([i for i in returned_fixed.values()])
	147	+ all_fin = sum([i for i in finished.values()])
	148	+ print fill("Suma"), fill(str(all_curr)), fill(str(all_ret) + " (" + str(all_ret_fixed) + ")"), fill(str(all_fin)), fill(str(all - all_curr - all_fin - all_ret))
	149	+ print ""
	150	+ print "Pozostalo do pobrania naprawionych:", all_ret_fixed
	151	+ print "Pozostalo do obejrzenia zwroconych:", all_ret - all_ret_fixed
	152	+ print
	153	+
	154	+if __name__ == "__main__":
	155	+ optparser = OptionParser(usage="""usage: %prog CONFIG""")
	156	+ (options, args) = optparser.parse_args()
	157	+ if len(args) < 1:
	158	+ optparser.print_help()
	159	+ sys.exit(0)
	160	+
	161	+ conf_path = args[0]
	162	+ cfg = Config(conf_path)
	163	+ wc = Repo(cfg["svn.repository"], cfg["svn.login"], cfg["svn.passwd"])
	164	+ db = Database(wc.db_path(), int(cfg["anno_per_file"]))
	165	+ anno_per_file = int(cfg["anno_per_file"])
	166	+
	167	+ rejected = count_rejected(db)
	168	+ print
	169	+ print fill_custom("", 100, "-")
	170	+ all = len(db.file_index.keys())
	171	+ print "Wszystkich tekstow w bazie: " + str(all)
	172	+ print "Odrzuconych: " + str(rejected)
	173	+ print "Wszystkich tekstow w bazie bez odrzuconych: " + str(all - rejected)
	174	+ all = all * anno_per_file - rejected
	175	+ print "Anotacji do wykonania: " + str(all)
	176	+
	177	+ print fill_custom("", 100, "-")
	178	+ print
	179	+
	180	+ print fill_custom("----Anotacja", 100, "-")
	181	+ anno_count = count_anno_stats(db, all, anno_per_file)
	182	+
	183	+ print fill_custom("----SuperAnotacja", 100, "-")
	184	+ count_super_stats(db, anno_count, anno_per_file)
	185	+
	186	+
...	...

scripts/extract_files.py 0 → 100755

View file @44cbbee

	1	+++ a/scripts/extract_files.py
	1	+#!/usr/bin/env python
	2	+
	3	+import sys
	4	+import os
	5	+import re
	6	+import shutil
	7	+from collections import defaultdict
	8	+from optparse import OptionParser
	9	+
	10	+# Solution with no hard coded path would be welcome
	11	+sys.path.append(os.path.join(os.path.dirname(os.path.abspath(sys.argv[0])), ".."))
	12	+
	13	+from dfs.database import Database
	14	+from dfs.repo import Repo
	15	+from dfs.config import Config
	16	+
	17	+def extract_file(file, exts, target_dir):
	18	+ print "Extracting file", file
	19	+ for ext in exts:
	20	+ filename = file + ext
	21	+ print "\t-", os.path.basename(filename)
	22	+ shutil.copy(filename, target_dir)
	23	+
	24	+if __name__ == "__main__":
	25	+ optparser = OptionParser(usage="""usage: %prog [options] CONFIG TYPE TARGET_DIR""")
	26	+ optparser.add_option("--extensions", dest="exts", default=".mmax,_mentions.xml,_words.xml",
	27	+ help="List of comma-separated file extensions")
	28	+ (options, args) = optparser.parse_args()
	29	+ if len(args) != 3:
	30	+ optparser.print_help()
	31	+ sys.exit(0)
	32	+
	33	+ conf_path = args[0]
	34	+ cfg = Config(conf_path)
	35	+ type = args[1]
	36	+ target_dir = args[2]
	37	+ anno_per_file = int(cfg["anno_per_file"])
	38	+ exts = options.exts.split(",")
	39	+ wc = Repo(cfg["svn.repository"], cfg["svn.login"], cfg["svn.passwd"])
	40	+ db = Database(wc.db_path(), anno_per_file)
	41	+
	42	+ poss_types = ["finished", "returned", "annotated"]
	43	+ if type not in poss_types:
	44	+ print "Possible types: ", poss_types
	45	+ sys.exit(0)
	46	+
	47	+ filepaths = {}
	48	+ for filename, file_elem in db.file_index.iteritems():
	49	+ if db.rejected(file_elem):
	50	+ continue
	51	+
	52	+ if type == "finished":
	53	+ sann = file_elem.find("s_ann")
	54	+ if sann is not None and db.finished(sann):
	55	+ filepaths[wc.upload_prim_path(filename)] = None
	56	+
	57	+ elif type == "annotated":
	58	+ idx = 0
	59	+ for ann in file_elem.findall("ann"):
	60	+ if db.finished(ann):
	61	+ filepaths[wc.upload_path(filename, idx)] = None
	62	+ idx = idx + 1
	63	+
	64	+ elif type == "returned":
	65	+ reason = None
	66	+ idx = 0
	67	+ for ann in file_elem.findall("ann"):
	68	+ if db.returned(ann):
	69	+ reason = db.get_reason(ann)
	70	+ break
	71	+ idx = idx + 1
	72	+
	73	+ if reason is not None:
	74	+ filepaths[wc.upload_path(filename, idx)] = reason
	75	+ continue
	76	+
	77	+ reason = None
	78	+ for sann in file_elem.findall("s_ann"):
	79	+ if db.returned(sann):
	80	+ reason = db.get_reason(sann)
	81	+ break
	82	+
	83	+ if reason is not None:
	84	+ filepaths[wc.upload_prim_path(filename)] = reason
	85	+
	86	+ for k, v in filepaths.iteritems():
	87	+ extract_file(k, exts, target_dir)
	88	+
	89	+ if type == "returned":
	90	+ filename = os.path.join(target_dir, "reasons.txt")
	91	+ with open(filename, 'w') as f:
	92	+ for k, v in sorted(filepaths.iteritems()):
	93	+ name = os.path.basename(k)
	94	+ f.write(name)
	95	+ f.write(" - ")
	96	+ f.write(v.encode("utf-8"))
	97	+ f.write("\n")
	98	+
0	99	\ No newline at end of file
...	...

scripts/fill_empty_repo.sh 0 → 100755

View file @44cbbee

	1	+++ a/scripts/fill_empty_repo.sh
	1	+#!/bin/bash
	2	+
	3	+if [ $# == 2 ]; then
	4	+ echo "Filling empty repository in: $1 with $2 annos per file"
	5	+else
	6	+ echo "Usage: $0 DIR ANNO_PER_FILE"
	7	+ exit
	8	+fi
	9	+
	10	+path=$1
	11	+annos=$2
	12	+
	13	+mkdir $path/new
	14	+mkdir $path/annotation
	15	+
	16	+c=0
	17	+for x in {A..Z}
	18	+do
	19	+ mkdir $path/annotation/$x
	20	+ c=$((c+1))
	21	+ if [ $c -ge $annos ]
	22	+ then
	23	+ break
	24	+ fi
	25	+done
	26	+
	27	+mkdir $path/adjudication
	28	+
	29	+echo "<files></files>" > $path/db.xml
...	...

scripts/fix_files.py 0 → 100755

View file @44cbbee

	1	+++ a/scripts/fix_files.py
	1	+#!/usr/bin/env python
	2	+
	3	+import sys
	4	+import os
	5	+import re
	6	+import shutil
	7	+from collections import defaultdict
	8	+from optparse import OptionParser
	9	+
	10	+# Solution with no hard coded path would be welcome
	11	+sys.path.append(os.path.join(os.path.dirname(os.path.abspath(sys.argv[0])), ".."))
	12	+
	13	+from dfs.database import Database
	14	+from dfs.repo import Repo
	15	+from dfs.config import Config
	16	+
	17	+def fix_file(wc, db, file_id, contents, anno_per_file):
	18	+ file_elem = db.file_index[file_id]
	19	+ if file_elem is None:
	20	+ print "File", file_id, "not found in database."
	21	+ return False
	22	+
	23	+ ann_elem = None
	24	+ idx = -1
	25	+ for ann in file_elem.findall("ann") + file_elem.findall("s_ann"):
	26	+ idx = idx + 1
	27	+ if db.returned(ann):
	28	+ ann_elem = ann
	29	+ break
	30	+
	31	+ if ann_elem is None:
	32	+ print "File", file_id, "not returned in database."
	33	+ return False
	34	+
	35	+ print "Fixing file " + file_id
	36	+
	37	+ try:
	38	+ db.fix(ann_elem)
	39	+
	40	+ if ann_elem.tag == "ann":
	41	+ wc.upload(file_id, idx, contents)
	42	+ elif ann_elem.tag == "s_ann":
	43	+ wc.upload_prim(file_id, contents)
	44	+
	45	+ except Exception as ex:
	46	+ print "\t error: " + str(ex)
	47	+ wc.revert()
	48	+ return False
	49	+
	50	+ return True
	51	+
	52	+def match_ext(path, exts):
	53	+ for ext in exts:
	54	+ if path.endswith(ext):
	55	+ return ext
	56	+
	57	+def path_id(path, ext):
	58	+ _, filename = os.path.split(path)
	59	+ return re.sub("%s$" % ext, "", filename)
	60	+
	61	+def group_paths(paths, exts):
	62	+ result = defaultdict(list)
	63	+ for path in paths:
	64	+ ext = match_ext(path, exts)
	65	+ if (ext != None):
	66	+ file_id = path_id(path, ext)
	67	+ if file_id not in result:
	68	+ result[file_id] = {}
	69	+ with open(path, "r") as f:
	70	+ result[file_id][ext]=f.read()
	71	+ return result
	72	+
	73	+def get_rec_paths(paths):
	74	+ result = []
	75	+ for path in paths:
	76	+ if os.path.isdir(path):
	77	+ for dirname, dirnames, filenames in os.walk(path):
	78	+ for filename in filenames:
	79	+ result.append(os.path.join(dirname, filename))
	80	+ else:
	81	+ result.append(path)
	82	+ return result
	83	+
	84	+if __name__ == "__main__":
	85	+ optparser = OptionParser(usage="""usage: %prog [options] CONFIG FILES""")
	86	+ optparser.add_option("--extensions", dest="exts", default=".mmax,_mentions.xml,_words.xml",
	87	+ help="List of comma-separated file extensions")
	88	+ (options, args) = optparser.parse_args()
	89	+ if len(args) < 2:
	90	+ optparser.print_help()
	91	+ sys.exit(0)
	92	+
	93	+ conf_path = args[0]
	94	+ cfg = Config(conf_path)
	95	+ anno_per_file = int(cfg["anno_per_file"])
	96	+ paths = get_rec_paths(args[1:])
	97	+ exts = options.exts.split(",")
	98	+ files = group_paths(paths, exts)
	99	+ wc = Repo(cfg["svn.repository"], cfg["svn.login"], cfg["svn.passwd"])
	100	+ db = Database(wc.db_path(), anno_per_file)
	101	+
	102	+ success = []
	103	+ fail = []
	104	+ for file_id, contents in files.iteritems():
	105	+ if fix_file(wc, db, file_id, contents, anno_per_file):
	106	+ success.append(file_id)
	107	+ else:
	108	+ fail.append(file_id)
	109	+
	110	+ db.save()
	111	+ wc.commit("Fixed files: " + str(success))
	112	+
	113	+ print ""
	114	+ if len(success) > 0:
	115	+ print "Fixed files: " + str(success)
	116	+ if len(fail) > 0:
	117	+ print "Failed to fix files: " + str(fail)
	118	+
...	...

scripts/quasi.py 0 → 100755

View file @44cbbee

	1	+++ a/scripts/quasi.py
	1	+#!/usr/bin/env python
	2	+import re
	3	+import sys
	4	+import os
	5	+import re
	6	+import shutil
	7	+from collections import defaultdict
	8	+from optparse import OptionParser
	9	+
	10	+# Solution with no hard coded path would be welcome
	11	+sys.path.append(os.path.join(os.path.dirname(os.path.abspath(sys.argv[0])), ".."))
	12	+
	13	+from dfs.database import Database
	14	+from dfs.repo import Repo
	15	+from dfs.config import Config
	16	+
	17	+def fill(text):
	18	+ return fill_custom(text, 20, " ")
	19	+
	20	+def fill_custom(text, l, sym):
	21	+ to_add = max(0, l - len(text))
	22	+ spaces = ""
	23	+ for i in range(to_add):
	24	+ spaces = spaces + sym
	25	+ return text + spaces
	26	+
	27	+def parse_span(span, words_list):
	28	+ words = []
	29	+
	30	+ w = span.split(",")
	31	+ for fragment in w:
	32	+ f = fragment.split("..")
	33	+ id1 = words_list.index(f[0])
	34	+ id2 = words_list.index(f[-1])
	35	+ for i in range(id1, id2+1):
	36	+ words.append(words_list[i])
	37	+
	38	+ return words
	39	+
	40	+def get_context(ids, words_list, size):
	41	+ first = -1
	42	+ last = -1
	43	+
	44	+ i = 0
	45	+ for i in range(len(words_list)):
	46	+ w = words_list[i]
	47	+ if w in ids:
	48	+ if first == -1:
	49	+ first = i
	50	+ last = i
	51	+
	52	+ first = max(0, first - size)
	53	+ last = min(len(words_list), last + size)
	54	+
	55	+ return words_list[first:last]
	56	+
	57	+def print_file(path):
	58	+ word_id_2_orth = {}
	59	+ words_list = []
	60	+ w = re.compile("<word.* id=\"(.?)\".>(.?)</word>.")
	61	+ with open(path + "_words.xml", "r") as f:
	62	+ for line in f.readlines():
	63	+ groups = w.findall(line)
	64	+ if len(groups) == 1:
	65	+ group = groups[0]
	66	+ id = group[0]
	67	+ orth = group[1]
	68	+ word_id_2_orth[id] = orth
	69	+ words_list.append(id)
	70	+
	71	+ me = re.compile("<markable.id=\"(.?)\".*")
	72	+ sp = re.compile(".span=\"(.?)\".*")
	73	+ co = re.compile(".comment=\"(.?)\".*")
	74	+ ni = re.compile(".near_identity=\"(.?)\".*")
	75	+
	76	+ mention_id_2_span = {}
	77	+ mention_id_2_comment = {}
	78	+ near_links = []
	79	+ with open(path + "_mentions.xml", "r") as f:
	80	+ for line in f.readlines():
	81	+ groups1 = me.findall(line)
	82	+ groups2 = sp.findall(line)
	83	+ groups3 = co.findall(line)
	84	+
	85	+ if len(groups1) == 1 and len(groups2) == 1:
	86	+ id = groups1[0]
	87	+ span = groups2[0]
	88	+ mention_id_2_span[id] = parse_span(span, words_list)
	89	+
	90	+ if len(groups3) == 1:
	91	+ mention_id_2_comment[id] = groups3[0]
	92	+
	93	+ near = ni.findall(line)
	94	+ if len(near) == 1 and near[0] != "empty" and near[0] != "":
	95	+ near_links.append((id, near[0]))
	96	+
	97	+ if len(near_links) > 0:
	98	+ print "###", "Tekst", path.split("/")[-1], "###"
	99	+ print
	100	+
	101	+ c = 0
	102	+ for m1, m2 in near_links:
	103	+ comments = []
	104	+ if m1 in mention_id_2_comment:
	105	+ comments.append(mention_id_2_comment[m1])
	106	+ if m2 in mention_id_2_comment:
	107	+ comments.append(mention_id_2_comment[m2])
	108	+
	109	+ if m1 not in mention_id_2_span or m2 not in mention_id_2_span:
	110	+ print "ERROR", m1, m2
	111	+ continue
	112	+
	113	+ span1 = mention_id_2_span[m1]
	114	+ span2 = mention_id_2_span[m2]
	115	+
	116	+ spans = set(span1)
	117	+ spans = spans.union(set(span2))
	118	+ ctx = get_context(spans, words_list, 3)
	119	+
	120	+ result = ""
	121	+ for wid in ctx:
	122	+ result = result + " "
	123	+ if wid == span1[0]:
	124	+ result = result + "["
	125	+ if wid == span2[0]:
	126	+ result = result + "["
	127	+
	128	+ result = result + word_id_2_orth[wid]
	129	+
	130	+ if wid == span1[-1]:
	131	+ result = result + "]"
	132	+ if wid == span2[-1]:
	133	+ result = result + "]"
	134	+
	135	+ m1orth = "[" + reduce(lambda a, i: a + " " + i, map(lambda i : word_id_2_orth[i], span1)) + "]"
	136	+ m2orth = "[" + reduce(lambda a, i: a + " " + i, map(lambda i : word_id_2_orth[i], span2)) + "]"
	137	+
	138	+ if words_list.index(span1[0]) <= words_list.index(span2[0]):
	139	+ print str(c)+".", m1orth, "<-->", m2orth
	140	+ else:
	141	+ print str(c)+".", m2orth, "<-->", m1orth
	142	+
	143	+ for comm in comments:
	144	+ print "Komentarz:", comm
	145	+ print "...", result, "..."
	146	+ print
	147	+
	148	+ c = c + 1
	149	+
	150	+ return c
	151	+
	152	+def print_quasi(db):
	153	+ links = 0
	154	+ for filename, file in db.file_index.iteritems():
	155	+
	156	+ if db.rejected(file):
	157	+ continue
	158	+
	159	+ sann = file.find("s_ann")
	160	+
	161	+ if sann is not None and db.finished(sann):
	162	+ path = wc.upload_prim_path(filename)
	163	+ links = links + print_file(path)
	164	+
	165	+ return links
	166	+
	167	+if __name__ == "__main__":
	168	+ optparser = OptionParser(usage="""usage: %prog CONFIG """)
	169	+ (options, args) = optparser.parse_args()
	170	+ if len(args) < 1:
	171	+ optparser.print_help()
	172	+ sys.exit(0)
	173	+
	174	+ conf_path = args[0]
	175	+ cfg = Config(conf_path)
	176	+ wc = Repo(cfg["svn.repository"], cfg["svn.login"], cfg["svn.passwd"])
	177	+ db = Database(wc.db_path(), int(cfg["anno_per_file"]))
	178	+
	179	+ l = print_quasi(db)
	180	+ print "##################"
	181	+ print
	182	+ print "Wszystkich linkow:", l
	183	+
...	...

scripts/reject_files.py 0 → 100755

View file @44cbbee

	1	+++ a/scripts/reject_files.py
	1	+#!/usr/bin/env python
	2	+
	3	+import sys
	4	+import os
	5	+import re
	6	+import shutil
	7	+from collections import defaultdict
	8	+from optparse import OptionParser
	9	+
	10	+# Solution with no hard coded path would be welcome
	11	+sys.path.append(os.path.join(os.path.dirname(os.path.abspath(sys.argv[0])), ".."))
	12	+
	13	+from dfs.database import Database
	14	+from dfs.repo import Repo
	15	+from dfs.config import Config
	16	+
	17	+def get_reason(db, file_id):
	18	+ reason = ""
	19	+ file_elem = db.file_index[file_id]
	20	+ for ann in file_elem.findall("ann"):
	21	+ if db.returned(ann):
	22	+ reason = db.get_reason(ann)
	23	+ break
	24	+
	25	+ for ann in file_elem.findall("s_ann"):
	26	+ if db.returned(ann):
	27	+ reason = db.get_reason(ann)
	28	+ break
	29	+
	30	+ return reason
	31	+
	32	+
	33	+def reject_file(db, file_id, reason):
	34	+ try:
	35	+ if reason == "":
	36	+ reason = get_reason(db, file_id)
	37	+
	38	+ print "Rejecting", file_id, "because", reason
	39	+ db.reject(file_id, reason)
	40	+ except Exception as ex:
	41	+ print file_id, "-", str(ex)
	42	+ return False
	43	+
	44	+ return True
	45	+
	46	+if __name__ == "__main__":
	47	+ optparser = OptionParser(usage="""usage: %prog -r REASON CONFIG FILES""")
	48	+ optparser.add_option("-r", dest="reason", default="", help="Reason to reject files")
	49	+ (options, args) = optparser.parse_args()
	50	+ if len(args) < 2:
	51	+ optparser.print_help()
	52	+ sys.exit(0)
	53	+
	54	+ conf_path = args[0]
	55	+ cfg = Config(conf_path)
	56	+ ids = args[1:]
	57	+ reason = options.reason
	58	+ anno_per_file = int(cfg["anno_per_file"])
	59	+ wc = Repo(cfg["svn.repository"], cfg["svn.login"], cfg["svn.passwd"])
	60	+ db = Database(wc.db_path(), anno_per_file)
	61	+
	62	+ success = []
	63	+ fail = []
	64	+ for file_id in ids:
	65	+ if reject_file(db, file_id, reason):
	66	+ success.append(file_id)
	67	+ else:
	68	+ fail.append(file_id)
	69	+
	70	+ db.save()
	71	+ wc.commit("Rejected files: "+str(ids))
	72	+
	73	+ print ""
	74	+ if len(success) > 0:
	75	+ print "Rejected files: "+str(success)
	76	+ if len(fail) > 0:
	77	+ print "Failed to reject files: "+str(fail)
	78	+
0	79	\ No newline at end of file
...	...

scripts/remove_files.py 0 → 100755

View file @44cbbee

	1	+++ a/scripts/remove_files.py
	1	+#!/usr/bin/env python
	2	+
	3	+import sys
	4	+import os
	5	+import re
	6	+import shutil
	7	+from collections import defaultdict
	8	+from optparse import OptionParser
	9	+
	10	+# Solution with no hard coded path would be welcome
	11	+sys.path.append(os.path.join(os.path.dirname(os.path.abspath(sys.argv[0])), ".."))
	12	+
	13	+from dfs.database import Database
	14	+from dfs.repo import Repo
	15	+from dfs.config import Config
	16	+
	17	+def remove_file(wc, db, file_id, extensions, anno_per_file):
	18	+ try:
	19	+ db.remove(file_id)
	20	+ except Exception as ex:
	21	+ print str(ex)
	22	+ return False
	23	+
	24	+ print "Removing "+file_id+" :"
	25	+ for ext in extensions:
	26	+ file = file_id+ext
	27	+ for r in wc.remove(file, anno_per_file):
	28	+ print "\t - "+r
	29	+
	30	+ return True
	31	+
	32	+if __name__ == "__main__":
	33	+ optparser = OptionParser(usage="""usage: %prog [options] CONFIG FILES""")
	34	+ optparser.add_option("--extensions", dest="exts", default=".mmax,_mentions.xml,_words.xml",
	35	+ help="List of comma-separated file extensions")
	36	+ (options, args) = optparser.parse_args()
	37	+ if len(args) < 2:
	38	+ optparser.print_help()
	39	+ sys.exit(0)
	40	+
	41	+ conf_path = args[0]
	42	+ cfg = Config(conf_path)
	43	+ ids = args[1:]
	44	+ extensions = options.exts.split(",")
	45	+ anno_per_file = int(cfg["anno_per_file"])
	46	+ wc = Repo(cfg["svn.repository"], cfg["svn.login"], cfg["svn.passwd"])
	47	+ db = Database(wc.db_path(), anno_per_file)
	48	+
	49	+ success = []
	50	+ fail = []
	51	+ for file_id in ids:
	52	+ if remove_file(wc, db, file_id, extensions, anno_per_file):
	53	+ success.append(file_id)
	54	+ else:
	55	+ fail.append(file_id)
	56	+
	57	+ db.save()
	58	+ wc.commit("Removed files: "+str(ids))
	59	+
	60	+ print ""
	61	+ if len(success) > 0:
	62	+ print "Removed files: "+str(success)
	63	+ if len(fail) > 0:
	64	+ print "Failed to remove files: "+str(fail)
0	65	\ No newline at end of file
...	...

scripts/stats_atlas.py 0 → 100755

View file @44cbbee

	1	+++ a/scripts/stats_atlas.py
	1	+#!/usr/bin/env python
	2	+
	3	+import sys
	4	+import os
	5	+import re
	6	+import shutil
	7	+from collections import defaultdict
	8	+from optparse import OptionParser
	9	+
	10	+# Solution with no hard coded path would be welcome
	11	+sys.path.append(os.path.join(os.path.dirname(os.path.abspath(sys.argv[0])), ".."))
	12	+
	13	+from dfs.database import Database
	14	+from dfs.repo import Repo
	15	+from dfs.config import Config
	16	+
	17	+def fill(text):
	18	+ return fill_custom(text, 20, " ")
	19	+
	20	+def fill_custom(text, l, sym):
	21	+ to_add = max(0, l - len(text))
	22	+ spaces = ""
	23	+ for i in range(to_add):
	24	+ spaces = spaces + sym
	25	+ return text + spaces
	26	+
	27	+def count_rejected(db):
	28	+ rejected = 0
	29	+ for filename, file in db.file_index.iteritems():
	30	+ if db.rejected(file):
	31	+ rejected = rejected + 1
	32	+ return rejected
	33	+
	34	+
	35	+def count_file_stats(path):
	36	+ text = ""
	37	+ with open(path + ".txt", "r") as f:
	38	+ for line in f.readlines():
	39	+ if line.startswith("#### SUMMARIES ####"):
	40	+ break;
	41	+ text += line +"\n"
	42	+
	43	+ return len(text)
	44	+
	45	+def count_anno_stats(db, all, anno_per_file):
	46	+ finished = {}
	47	+ current = {}
	48	+ returned = {}
	49	+ returned_fixed = {}
	50	+ finished_by_all = 0
	51	+ fetched_by_all = 0
	52	+ chars = {}
	53	+
	54	+ for filename, file in db.file_index.iteritems():
	55	+ if db.rejected(file):
	56	+ continue
	57	+
	58	+ fin = True
	59	+ idx = 0
	60	+ for annotation in file.findall("ann"):
	61	+ owner = annotation.find("annName").text
	62	+
	63	+ if not owner in finished:
	64	+ finished[owner] = 0
	65	+ if not owner in current:
	66	+ current[owner] = 0
	67	+ if not owner in returned:
	68	+ returned[owner] = 0
	69	+ if not owner in returned_fixed:
	70	+ returned_fixed[owner] = 0
	71	+
	72	+ if db.owned(annotation):
	73	+ current[owner] = current[owner] + 1
	74	+ elif db.returned(annotation):
	75	+ returned[owner] = returned[owner] + 1
	76	+ if db.fixed(annotation):
	77	+ returned_fixed[owner] = returned_fixed[owner] + 1
	78	+ elif db.finished(annotation):
	79	+ finished[owner] = finished[owner] + 1
	80	+ else:
	81	+ print "Strange state! ", annotation.text
	82	+
	83	+ if not db.finished(annotation):
	84	+ fin = False
	85	+ else:
	86	+ path = wc.upload_path(filename, idx)
	87	+ ccnt = count_file_stats(path)
	88	+ if owner not in chars:
	89	+ chars[owner] = 0
	90	+ chars[owner] = chars[owner] + ccnt
	91	+
	92	+ if len(file.findall("ann")) == anno_per_file:
	93	+ if fin:
	94	+ finished_by_all = finished_by_all + 1
	95	+ else:
	96	+ fetched_by_all = fetched_by_all + 1
	97	+
	98	+ idx += 1
	99	+
	100	+
	101	+ print fill("Login"), fill("Akt. pobr."), fill("Zwrocone (Napr.)"), fill("Zakonczone"), fill("Zak. znaki"), fill("Do pobrania")
	102	+ print fill_custom("", 120, "-")
	103	+ for user in set(current.keys()) \| set(finished.keys()) \| set(returned_fixed.keys()) \| set(returned.keys()):
	104	+ print fill(user), fill(str(current[user])), fill(str(returned[user]) + " (" + str(returned_fixed[user]) + ")"), fill(str(finished[user])),
	105	+ print fill(str(chars[user])),
	106	+ priority, normal = db.for_annotation(user)
	107	+ print fill(str(len(normal)))
	108	+
	109	+ print fill_custom("", 120, "-")
	110	+ all_curr = sum([i for i in current.values()])
	111	+ all_ret = sum([i for i in returned.values()])
	112	+ all_ret_fixed = sum([i for i in returned_fixed.values()])
	113	+ all_fin = sum([i for i in finished.values()])
	114	+ all_chars = sum([i for i in chars.values()])
	115	+ print fill("Suma"), fill(str(all_curr)), fill(str(all_ret) + " (" + str(all_ret_fixed) + ")"), fill(str(all_fin)), fill(str(all_chars)), fill(str(all - all_curr - all_fin - all_ret))
	116	+ print ""
	117	+ print "Pobranych (nie zakonczonych) jednoczesnie przez", anno_per_file, "tekstow:", fetched_by_all
	118	+ print "Zakonczonych jednoczesnie przez", anno_per_file, "tekstow:", finished_by_all
	119	+ print "Pozostalo do pobrania naprawionych:", all_ret_fixed
	120	+ print "Pozostalo do obejrzenia zwroconych:", all_ret - all_ret_fixed
	121	+ print
	122	+
	123	+ return finished_by_all
	124	+
	125	+if __name__ == "__main__":
	126	+ optparser = OptionParser(usage="""usage: %prog CONFIG""")
	127	+ (options, args) = optparser.parse_args()
	128	+ if len(args) < 1:
	129	+ optparser.print_help()
	130	+ sys.exit(0)
	131	+
	132	+ conf_path = args[0]
	133	+ cfg = Config(conf_path)
	134	+ wc = Repo(cfg["svn.repository"], cfg["svn.login"], cfg["svn.passwd"])
	135	+ db = Database(wc.db_path(), int(cfg["anno_per_file"]))
	136	+ anno_per_file = int(cfg["anno_per_file"])
	137	+
	138	+ rejected = count_rejected(db)
	139	+ print
	140	+ print fill_custom("", 100, "-")
	141	+ all = len(db.file_index.keys())
	142	+ print "Wszystkich tekstow w bazie: " + str(all)
	143	+ print "Odrzuconych: " + str(rejected)
	144	+ print "Wszystkich tekstow w bazie bez odrzuconych: " + str(all - rejected)
	145	+ all = all * anno_per_file - rejected
	146	+ print "Anotacji do wykonania: " + str(all)
	147	+
	148	+ print fill_custom("", 100, "-")
	149	+ print
	150	+
	151	+ print fill_custom("----Anotacja", 100, "-")
	152	+ anno_count = count_anno_stats(db, all, anno_per_file)
	153	+
0	154	\ No newline at end of file
...	...