
File processing
Input format: Input files must be in .txt FORMAT with UTF-8 ENCODING and contain PORTUGUESE TEXT. Input files and folders can also be compressed to the .zip format.
Privacy: The input file you upload and the respective output files will be automatically deleted from our computer after being processed and the result downloaded by you. No copies of your files will be retained after your use of this service.
Email address validation
Loading...
The size of your input file is large and its processing may take some time.
To receive by email an URL from which to download your processed file, please copy the code displayed below into the field "Subject:" of an email message (with the message body empty) and send it to request@portulanclarin.net
To proceed, please send an email to request@portulanclarin.net with the following code in the "Subject" field:
To: | request@portulanclarin.net |
|
Subject: |
|
The communication with the server cannot be established. Please try again later.
We are sorry but an unexpected error has occurred. Please try again later.
The code has expired. Please click the button below to get a new code.
For enhanced security, a new code has to be validated. Please click the button below to get a new code.
Privacy: After we reply to you with the URL for download, your email address is automatically deleted from our records.
Designing your own experiment with a Jupyter Notebook
A Jupyter notebook (hereafter just notebook, for short) is a type of document that contains executable code interspersed with visualizations of code execution results and narrative text.
Below we provide an example notebook which you may use as a starting point for designing your own experiments using language resources offered by PORTULAN CLARIN.
Pre-requisites
To execute this notebook, you need an access key you can obtain by clicking the button below. A key is valid for 31 days. It allows to submit a total of 500 million characters by means of requests with no more 2000 characters each. It allows to enter 100,000 requests, at a rate of no more than 200 requests per hour.
For other usage regimes, you should contact the helpdesk.
The input data sent to any PORTULAN CLARIN web service and the respective output will be automatically deleted from our computers after being processed. However, when running a notebook on an external service, such as the ones suggested below, you should take their data privacy policies into consideration.
Running the notebook
You have three options to run the notebook presented below:
- Run on Binder — The Binder Project is funded by a 501c3 non-profit
organization and is described in detail in the following paper:
Jupyter et al., "Binder 2.0 - Reproducible, Interactive, Sharable Environments for Science at Scale."
Proceedings of the 17th Python in Science Conference. 2018. doi://10.25080/Majora-4af1f417-011 - Run on Google Colab — Google Colaboratory is a free-to-use product from Google Research.
- Download the notebook from our public Github repository and run it on your computer.
This is a more advanced option, which requires you to install Python 3 and Jupyter on your computer. For anyone without prior experience setting up a Python development environment, we strongly recommend one of the two options above.
This is only a preview of the notebook. To run it, please choose one of the following options:
Using LX-UDParser to parse sentences and displaying dependency tree graphs¶
This is an example notebook that illustrates how you can use the LX-UDParser web service to parse sentences and how to visualize dependency tree graphs in a notebook.
Before you run this example, replace access_key_goes_here
by your webservice access key, below:
LXUDPARSER_WS_API_KEY = 'access_key_goes_here'
LXUDPARSER_WS_API_KEY = 'a38fae3bea4ef5b676279a02cacd8ca5'
LXUDPARSER_WS_API_URL = 'https://portulanclarin.net/workbench/lx-udparser/api/'
Importing required Python modules¶
The next cell will take care of installing the requests
and pydependencygrapher
packages,
if not already installed, and make them available to use in this notebook.
try:
import requests
except:
!pip3 install requests
import requests
try:
import pydependencygrapher
except:
# see https://github.com/pygobject/pycairo/issues/39#issuecomment-391830334
!apt-get install libcairo2-dev libjpeg-dev libgif-dev
!pip3 install pydependencygrapher
import pydependencygrapher
import base64
import IPython
Wrapping the complexities of the JSON-RPC API in a simple, easy to use function¶
The WSException
class defined below, will be used later to identify errors
from the webservice.
class WSException(Exception):
'Webservice Exception'
def __init__(self, errordata):
"errordata is a dict returned by the webservice with details about the error"
super().__init__(self)
assert isinstance(errordata, dict)
self.message = errordata["message"]
# see https://json-rpc.readthedocs.io/en/latest/exceptions.html for more info
# about JSON-RPC error codes
if -32099 <= errordata["code"] <= -32000: # Server Error
if errordata["data"]["type"] == "WebServiceException":
self.message += f": {errordata['data']['message']}"
else:
self.message += f": {errordata['data']!r}"
def __str__(self):
return self.message
The next function invoques the LX-UDParser webservice through it's public JSON-RPC API.
def parse(text, format):
'''
Arguments
text: a string with a maximum of 2000 characters, Portuguese text, with
the input to be processed
format: either 'CONLL' or 'JSON'
Returns a string with the output according to specification in
https://portulanclarin.net/workbench/lx-udparser/
Raises a WSException if an error occurs.
'''
request_data = {
'method': 'parse',
'jsonrpc': '2.0',
'id': 0,
'params': {
'text': text,
'format': format,
'key': LXUDPARSER_WS_API_KEY,
},
}
request = requests.post(LXUDPARSER_WS_API_URL, json=request_data)
response_data = request.json()
if "error" in response_data:
raise WSException(response_data["error"])
else:
return response_data["result"]
Let us test the function we just defined:
text = '''Esta frase serve para testar o funcionamento do parser de dependências. Esta outra
frase faz o mesmo.'''
# the CONLL annotation format is a popular format for annotating part of speech
# and dependency tree graphs
result = parse(text, format="CONLL")
print(result)
#id form lemma cpos pos feat head deprel phead pdeprel 1 Esta - DET DET Gender=Fem|Number=Sing 2 det 2 det 2 frase FRASE NOUN NOUN Gender=Fem|Number=Sing 3 nsubj 3 nsubj 3 serve SERVIR VERB VERB Tense=Pres|Mood=Ind|Person=3|Number=Sing 0 root 0 root 4 para - ADP ADP - 3 obj 3 obj 5 testar TESTAR VERB VERB - 3 dep 3 dep 6 o - DET DET Gender=Masc|Number=Sing 7 obj 7 obj 7 funcionamento FUNCIONAMENTO NOUN NOUN Gender=Masc|Number=Sing 5 pobj 5 pobj 8 de_ - ADP ADP - 7 fixed 7 fixed 9 o - DET DET Gender=Masc|Number=Sing 7 fixed 7 fixed 10 parser PARSER NOUN NOUN Gender=Masc|Number=Sing 7 pobj 7 pobj 11 de - ADP ADP - 10 fixed 10 fixed 12 dependências DEPENDÊNCIA NOUN NOUN Gender=Fem|Number=Plur 10 fixed 10 fixed 13 . - PUNCT PUNCT - 3 punct 3 punct #id form lemma cpos pos feat head deprel phead pdeprel 1 Esta - DET DET Gender=Fem|Number=Sing 3 det 3 det 2 outra OUTRO ADJ ADJ Gender=Fem|Number=Sing 3 det 3 det 3 frase FRASE NOUN NOUN Gender=Fem|Number=Sing 4 nsubj 4 nsubj 4 faz FAZER VERB VERB Tense=Pres|Mood=Ind|Person=3|Number=Sing 0 root 0 root 5 o - DET DET Gender=Masc|Number=Sing 4 dep 4 dep 6 mesmo MESMO ADJ ADJ Gender=Masc|Number=Sing 5 dep 5 dep 7 . - PUNCT PUNCT - 4 punct 4 punct
Displaying dependency tree graphs from parsed text in CONLL format¶
To view dependency tree graphs for the parsed sentences, first we will split the CONLL output on empty lines to get one set of lines per sentence (each line carrying information pertaining to each token).
def group_sentence_conll_lines(conll_lines):
"""Groups CONLL-encoded lines (one line encodes one token), according to sentences.
This generator function takes as argument a sequence of CONLL lines, and generates
a sequence of lists, each one containing the CONLL lines of a sentence
"""
parsed_sentences = []
current_sentence = []
for line in conll_lines:
# lines starting with # are comments; ignore
if line.startswith("#"):
continue
# one or more consecutive empty lines mark the end of a sentence
if not line:
if current_sentence:
parsed_sentences.append(current_sentence)
current_sentence = []
else:
current_sentence.append(line)
if current_sentence:
parsed_sentences.append(current_sentence)
return parsed_sentences
Let us define a function render_tree
that displays a sentence dependency graph, making use of the pydependencygrapher
package for rendering the graph into an image and the IPython
package for displaying the resulting image.
We also define a function render_tree_from_conll
that will take a CONLL sentence (a list of CONLL-formatted lines, one for each token) and create one pydependencygrapher.Token
object for each token, before calling render_tree
to display the dependency graph.
def render_tree(sentence):
graph = pydependencygrapher.DependencyGraph(sentence, show_tags=False)
graph.draw()
b64png = graph.save_buffer()
IPython.display.display(IPython.display.Image(data=base64.b64decode(b64png)))
def render_tree_from_conll(conll_sentence):
sentence = [pydependencygrapher.Token(*conll_token.split("\t")) for conll_token in conll_sentence]
return render_tree(sentence)
conll_lines = result.splitlines(keepends=False)
for conll_sentence in group_sentence_conll_lines(conll_lines):
data = render_tree_from_conll(conll_sentence)
The JSON output format¶
The JSON format (which we obtain by passing format="JSON"
into the parse
function) is more
convenient when we need to further process the annotations, because each abstraction is mapped
directly into a Python native object (lists, dicts, strings, etc) as follows:
- The returned object is a
list
, where each element corresponds to a paragraph of the given text; - In turn, each paragraph is a
list
where each element represents a sentence; - Each sentence is a
list
where each element represents a token; - Each token is a
dict
where each key-value pair is an attribute of the token.
parsed_text = parse(text, format="JSON")
for pnum, paragraph in enumerate(parsed_text, start=1): # enumerate paragraphs in text, starting at 1
print(f"paragraph {pnum}:")
for snum, sentence in enumerate(paragraph, start=1): # enumerate sentences in paragraph, starting at 1
print(f" sentence {snum}:")
for tnum, token in enumerate(sentence, start=1): # enumerate tokens in sentence, starting at 1
print(f" token {tnum}: {token!r}") # print a token representation
paragraph 1: sentence 1: token 1: {'form': 'Esta', 'space': 'LR', 'pos': 'DEM', 'upos': 'DET', 'feats': 'fs', 'ufeats': 'Gender=Fem|Number=Sing', 'udeprel': 'det', 'uparent': 2} token 2: {'form': 'frase', 'space': 'LR', 'pos': 'CN', 'upos': 'NOUN', 'lemma': 'FRASE', 'ulemma': 'FRASE', 'feats': 'fs', 'ufeats': 'Gender=Fem|Number=Sing', 'udeprel': 'nsubj', 'uparent': 3} token 3: {'form': 'serve', 'space': 'LR', 'pos': 'V', 'upos': 'VERB', 'lemma': 'SERVIR', 'ulemma': 'SERVIR', 'feats': 'pi-3s', 'ufeats': 'Tense=Pres|Mood=Ind|Person=3|Number=Sing', 'udeprel': 'root', 'uparent': 0} token 4: {'form': 'para', 'space': 'LR', 'pos': 'PREP', 'upos': 'ADP', 'udeprel': 'obj', 'uparent': 3} token 5: {'form': 'testar', 'space': 'LR', 'pos': 'V', 'upos': 'VERB', 'lemma': 'TESTAR', 'ulemma': 'TESTAR', 'feats': 'INF-nInf', 'udeprel': 'dep', 'uparent': 3} token 6: {'form': 'o', 'space': 'LR', 'pos': 'DA', 'upos': 'DET', 'feats': 'ms', 'ufeats': 'Gender=Masc|Number=Sing', 'udeprel': 'obj', 'uparent': 7} token 7: {'form': 'funcionamento', 'space': 'LR', 'pos': 'CN', 'upos': 'NOUN', 'lemma': 'FUNCIONAMENTO', 'ulemma': 'FUNCIONAMENTO', 'feats': 'ms', 'ufeats': 'Gender=Masc|Number=Sing', 'udeprel': 'pobj', 'uparent': 5} token 8: {'form': 'de_', 'space': 'L', 'raw': 'do', 'pos': 'PREP', 'upos': 'ADP', 'udeprel': 'fixed', 'uparent': 7} token 9: {'form': 'o', 'space': 'R', 'pos': 'DA', 'upos': 'DET', 'feats': 'ms', 'ufeats': 'Gender=Masc|Number=Sing', 'udeprel': 'fixed', 'uparent': 7} token 10: {'form': 'parser', 'space': 'LR', 'pos': 'CN', 'upos': 'NOUN', 'lemma': 'PARSER', 'ulemma': 'PARSER', 'feats': 'ms', 'ufeats': 'Gender=Masc|Number=Sing', 'udeprel': 'pobj', 'uparent': 7} token 11: {'form': 'de', 'space': 'LR', 'pos': 'PREP', 'upos': 'ADP', 'udeprel': 'fixed', 'uparent': 10} token 12: {'form': 'dependências', 'space': 'L', 'pos': 'CN', 'upos': 'NOUN', 'lemma': 'DEPENDÊNCIA', 'ulemma': 'DEPENDÊNCIA', 'feats': 'fp', 'ufeats': 'Gender=Fem|Number=Plur', 'udeprel': 'fixed', 'uparent': 10} token 13: {'form': '.', 'space': 'R', 'pos': 'PNT', 'upos': 'PUNCT', 'udeprel': 'punct', 'uparent': 3} sentence 2: token 1: {'form': 'Esta', 'space': 'LR', 'pos': 'DEM', 'upos': 'DET', 'feats': 'fs', 'ufeats': 'Gender=Fem|Number=Sing', 'udeprel': 'det', 'uparent': 3} token 2: {'form': 'outra', 'space': 'LR', 'pos': 'ADJ', 'upos': 'ADJ', 'lemma': 'OUTRO', 'ulemma': 'OUTRO', 'feats': 'fs', 'ufeats': 'Gender=Fem|Number=Sing', 'udeprel': 'det', 'uparent': 3} token 3: {'form': 'frase', 'space': 'LR', 'pos': 'CN', 'upos': 'NOUN', 'lemma': 'FRASE', 'ulemma': 'FRASE', 'feats': 'fs', 'ufeats': 'Gender=Fem|Number=Sing', 'udeprel': 'nsubj', 'uparent': 4} token 4: {'form': 'faz', 'space': 'LR', 'pos': 'V', 'upos': 'VERB', 'lemma': 'FAZER', 'ulemma': 'FAZER', 'feats': 'pi-3s', 'ufeats': 'Tense=Pres|Mood=Ind|Person=3|Number=Sing', 'udeprel': 'root', 'uparent': 0} token 5: {'form': 'o', 'space': 'LR', 'pos': 'LDEM1', 'upos': 'DET', 'ufeats': 'Gender=Masc|Number=Sing', 'udeprel': 'dep', 'uparent': 4} token 6: {'form': 'mesmo', 'space': 'L', 'pos': 'LDEM2', 'upos': 'ADJ', 'ulemma': 'MESMO', 'ufeats': 'Gender=Masc|Number=Sing', 'udeprel': 'dep', 'uparent': 5} token 7: {'form': '.', 'space': 'R', 'pos': 'PNT', 'upos': 'PUNCT', 'udeprel': 'punct', 'uparent': 4}
Displaying dependency graphs from parsed text in JSON format¶
Let us define a function, similar to render_tree_from_conll
to display dependency graphs for JSON-encoded sentences.
def render_tree_from_json(json_sentence):
token_attributes = ["form", "lemma", "upos", "upos", "ufeats", "uparent", "udeprel", "uparent", "udeprel"]
sentence = []
for num, token in enumerate(json_sentence, start=1):
sentence.append(
pydependencygrapher.Token(
num,
*[token.get(attribute, "_") for attribute in token_attributes]
)
)
return render_tree(sentence)
Let us test the function we just defined
for paragraph in parsed_text:
for sentence in paragraph:
render_tree_from_json(sentence)
Getting the status of a webservice access key¶
def get_key_status():
'''Returns a string with the detailed status of the webservice access key'''
request_data = {
'method': 'key_status',
'jsonrpc': '2.0',
'id': 0,
'params': {
'key': LXUDPARSER_WS_API_KEY,
},
}
request = requests.post(LXUDPARSER_WS_API_URL, json=request_data)
response_data = request.json()
if "error" in response_data:
raise WSException(response_data["error"])
else:
return response_data["result"]
get_key_status()
{'requests_remaining': 99989, 'chars_remaining': 499998825, 'expiry': '2022-03-05T02:12+00:00'}
Instructions to use this web service
The web service for this application is available at https://portulanclarin.net/workbench/lx-udparser/api/.
Below you find an example of how to use this web service with Python 3.
This example resorts to the requests package. To install this package, run this command in the command line:
pip3 install requests
.
To use this web service, you need an access key you can obtain by clicking in the button below. A key is valid for 31 days. It allows to submit a total of 500 million characters by means of requests with no more 2000 characters each. It allows to enter 100,000 requests, at a rate of no more than 200 requests per hour.
For other usage regimes, you should contact the helpdesk.
The input data and the respective output will be automatically deleted from our computer after being processed. No copies will be retained after your use of this service.
import json
import requests # to install this library, enter in your command line:
# pip3 install requests
# This is a simple example to illustrate how you can use the LX-UDParser web service
# Requires: key is a string with your access key
# Requires: text is a string, UTF-8, with a maximum 2000 characters, Portuguese text, with
# the input to be processed
# Requires: format is a string, indicating the output format, which can be either
# 'CONLL' or 'JSON'
# Ensures: output according to specification in https://portulanclarin.net/workbench/lx-udparser/
# Ensures: dict with number of requests and characters input so far with the access key, and
# its date of expiry
key = 'access_key_goes_here' # before you run this example, replace access_key_goes_here by
# your access key
# this string can be replaced by your input
text = '''A Maria tem razão.
Mesmo assim, ensaia algumas aproximações.
A emissão será cotada na Bolsa de Valores do Luxemburgo.'''
format = 'CONLL' # other possible value is 'JSON'
# To read input text from a file, uncomment this block
#inputFile = open("myInputFileName", "r", encoding="utf-8") # replace myInputFileName by
# the name of your file
#text = inputFile.read()
#inputFile.close()
# Processing:
url = "https://portulanclarin.net/workbench/lx-udparser/api/"
request_data = {
'method': 'parse',
'jsonrpc': '2.0',
'id': 0,
'params': {
'text': text,
'format': format,
'key': key,
},
}
request = requests.post(url, json=request_data)
response_data = request.json()
if "error" in response_data:
print("Error:", response_data["error"])
else:
print("Result:")
print(response_data["result"])
# To write output in a file, uncomment this block
#outputFile = open("myOutputFileName","w", encoding="utf-8") # replace myOutputFileName by
# the name of your file
#output = response_data["result"]
#outputFile.write(output)
#outputFile.close()
# Getting acess key status:
request_data = {
'method': 'key_status',
'jsonrpc': '2.0',
'id': 0,
'params': {
'key': key,
},
}
request = requests.post(url, json=request_data)
response_data = request.json()
if "error" in response_data:
print("Error:", response_data["error"])
else:
print("Key status:")
print(json.dumps(response_data["result"], indent=4))
Access key for the web service
This is your access key for this web service.
The following access key for this web service is already associated with .
This key is valid until and can be used to process requests or characters.
An email message has been sent into your address with the information above.
Email address validation
Loading...
To receive by email your access key for this webservice, please copy the code displayed below into the field "Subject" of an email message (with the message body empty) and send it to request@portulanclarin.net
To proceed, please send an email to request@portulanclarin.net with the following code in the "Subject" field:
To: | request@portulanclarin.net |
|
Subject: |
|
The communication with the server cannot be established. Please try again later.
We are sorry but an unexpected error has occurred. Please try again later.
The code has expired. Please click the button below to get a new code.
For enhanced security, a new code has to be validated. Please click the button below to get a new code.
Privacy: When your access key expires, your email address is automatically deleted from our records.
LX-UDParser's documentation
LX-UDParser
LX-UDParser is a free online service for the syntactic analysis of Portuguese. It allows the automatic parsing of sentences in Portuguese in terms of the grammatical functions of their words, following the Universal Dependencies framework.
This service was developed and is maintained at the University of Lisbon by the NLX-Speech and Natural Language Group, Department of Informatics.
Parser
LX-UDParser is a NLP4J model trained with Portuguese data.
For the training of the parser, 34,560 sentences were used (comprising 428k tokens). The sentences were taken from the CINTIL-UDep treebank. This treebank is being developed and maintained at the University of Lisbon by the NLX-Speech and Natural Language Group of the Department of Informatics. In terms of evaluation, LX-UDParser's UAS (unlabeled attachment score) is 91.11 and its LAS (labeled attachment score) is 90.31.
You may be interested to use also our LX-USuite, LX-UTagger, LX-DepParser, LX-Parser, LX-Suite or LX-Tagger online services.
Authorship
LX-UDParser was developed by João Silva and Luís Gomes, under the direction of António Branco at the NLX-Group on Natural Language and Speech.
Publications
Irrespective of the most recent version of this tool you may use, when mentioning it, please cite this reference:
- António Branco, João Silva, Luís Gomes, João Rodrigues, 2022, Universal Grammatical Dependencies for Portuguese with CINTIL Data, LX Processing and CLARIN support. (submitted)
Release
You can download the program here.
The CINTIL-UDep corpus, used for training the LX-UDParser model, is also available for download here
Contact us
You can contact us at the following email address: 'nlx' followed by '@' followed by 'di.fc.ul.pt'.
License
No fee, attribution, all rights reserved, no redistribution, non commercial, no warranty, no liability, no endorsement, temporary, non exclusive, share alike.
The complete text of this license is here.