This paper studies structure learning for DAGs and examines the role of sparsity and acyclic constraints in this problem. The paper shows that a regularized MLE objective recovers a DAG that is quasi-equivalent to the ground truth. The algorithm is also significantly more scalable than prior methods. This is a good step toward making structure learning in DAGs highly practical, and as such we recommend acceptance.