I alawys have trouble getting DyNet working with CUDA support. Here’s what worked today (Ubuntu 18.04 LTS; Linux 5.4.0; conda 4.9.2):
conda install -c conda-forge cudatoolkit-dev=10.2
conda install -c pytorch cudatoolkit=10.2
BACKEND=cuda pip install -v git+https://github.com/clab/dynet#egg=dynet