{"title": "Cascaded Dilated Dense Network with Two-step Data Consistency for MRI Reconstruction", "book": "Advances in Neural Information Processing Systems", "page_first": 1744, "page_last": 1754, "abstract": "Compressed Sensing MRI (CS-MRI) aims at reconstrcuting de-aliased images from sub-Nyquist sampling k-space data to accelerate MR Imaging. Inspired by recent deep learning methods, we propose a Cascaded Dilated Dense Network (CDDN) for MRI reconstruction. Dense blocks with residual connection are used to restore clear images step by step and dilated convolution is introduced for expanding receptive field without taking more network parameters. After each sub-network, we use a novel two-step Data Consistency (DC) operation in k-space. We convert the complex result from first DC operation to real-valued images and applied another sampled \\emph{k}-space data replacement. Extensive experiments demonstrate that the proposed CDDN with two-step DC achieves state-of-art result.", "full_text": "Cascaded Dilated Dense Network with Two-step Data\n\nConsistency for MRI Reconstruction\n\nHao Zheng, Faming Fang\u2217, Guixu Zhang\n\nShanghai Key Laboratory of Multidimensional Information Processing,\n\nand the School of Computer Science and Technology\n\nwsnbzh@hotmail.com, {fmfang, gxzhang}@cs.ecnu.edu.cn\n\nEast China Normal University\n\nAbstract\n\nCompressed Sensing MRI (CS-MRI) aims at reconstrcuting de-aliased images\nfrom sub-Nyquist sampling k-space data to accelerate MR Imaging. Inspired by\nrecent deep learning methods, we propose a Cascaded Dilated Dense Network\n(CDDN) for MRI reconstruction. Dense blocks with residual connection are used\nto restore clear images step by step and dilated convolution is introduced for\nexpanding receptive \ufb01eld without taking more network parameters. After each sub-\nnetwork, we use a novel Two-step Data Consistency (TDC) operation in k-space.\nWe convert the complex result from \ufb01rst DC operation to real-valued images and\napplied another replacement with sampled k-space data. Extensive experiments\ndemonstrate that the proposed CDDN with TDC achieves state-of-art result.\n\n1\n\nIntroduction\n\nMagnetic resonance imaging (MRI) [11] is widely used in clinical diagnosis. It extracts internal\ninformation of the human body to detect latent lesion. Unlike conventional imaging techniques, MRI\ngathers phase-encoding data from k-space instead of image domain. The scanning procecss should\nfollow the Nyquist criteria [14] to produce clear images, but it leads to long acquisition time. Patients\ncan have tension as they have to keep still in the entire process.\nSub-Nyquist sampling can signi\ufb01cantly reduce the acquisition time by skipping partial phase informa-\ntion, but it leads to aliased artifacts. In order to recovery clear image from sub-sampled k-space data,\nCS-MRI approaches were proposed [2]. With the assumption that MR images are sparsity in speci\ufb01c\ntransfrom domain, classic sparsity-prior methods apply transfroms like discrete Fourier transform\n(DFT) [5], discrete cosine transfrom (DCT) [20, 25] and discrete wavelet transform (DWT) [10, 16].\nData-driven methods (i.e. dictionary learning) achieve higher accuracy due to the adaptive feature\nrepresentation learnt from a quantity of fully sampled data [33]. Although these methods success in\nrestoring clear image, they still suffer from heavy computation overhead.\nRecent years, deep learning has achieved excellent result in a variety of image-restoring problem\nsuch as de-noising [34], de-blurring [28], super-resolution [26], etc. Generally, deep learning\nmethods develop deep neural network to learn the mapping function from one distribution to another\none. In MRI reconstruction, a common way is training a convolution neural network (CNN) for\nmapping from aliased images (directly reconstructed from zero-\ufb01lled sub-sampled k-space data) to\ncorresponding clear images [24]. U-Net is a popular framework in medical image processing [19].\nIt also accomplished accurate result on MRI reconstruction [7, 17, 29]. Yang et al. [30] proposed\nADMM-CSNet to learn the parameters of ADMM algorithm with nerual network instead of manual\nadjustment. More recently, cascading network was introduced to MRI reconstruction [21, 22].\n\n\u2217Corresponding Author\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fUnlike normal image restoration, original data of MRI is acquired from k-space. Frequency domain\ndata consistency plays an important role in MRI reconstruction. Hyun et al. [7] directly replaced the\ncorresponding phasing-encoding data with sampled data. Yang et al. [29] used frequency domain\nloss while Quan et al. [17] applied cyclic loss. Schlemper et al. [21] implemented a consistency layer\nwith a noise-adaptive parameter for noisy data. For cascading network, such data consistency layer\nprovides intermedia information correction between sub-networks.\nIn this paper, we propose a novel network architecture called Cascaded Dilated Dense Network with\nTwo-step Data Consistency layer (CDDNwithTDC). Our contributions can be summarized as follows:\n(1) We use cascaded dense blocks to reconstruct MR images to improve performance as well as\nreduce the number of parameters. Such intra-block dense shortcut architecture alleviates the gradient\nvanishment and preserves detail information.\n(2) We introduce dilated convolution to dense blocks, which expands receptive \ufb01eld without any\nadditional parameters. The combination suf\ufb01ciently extracts latent information.\n(3) We propose a Two-step Data Consistency layer to enhance the naturalness of MR images while\nensuring the data consistency in k-space.\nNumerous experiments show the advancement of our proposed method in MRI reconstruction. 2\n\n2 Related Works\n\n2.1 Cascaded Network\n\nCascaded network uses a serial of sub-networks to process data step by step. The later sub-networks\ntake the former result as input to improve the accuracy. Quan et al. [17] proposed Re\ufb01neGAN by\ncascading two U-Net as generator. As simply cascading network has no difference with naively in-\ncreasing the depth of network, it can easily reach a bottleneck of performance. In MRI reconstruction,\ndata consistency operation can be applied as a postprocess of sub-network, which replace the speci\ufb01c\nk-space position with the sampled value [22]. Such operation enable skip connection between input\nand each sub-network to alleviate gradient vanishment.\n\n2.2 Dense Connection\n\nIn general, a deeper network has higher performance, but it also suffers from gradient vanish problem.\nAfter a long chain of gradient backward , the gradient information in the early stage can be too small\nfor updating parameters. Skip connection alleviates such phenomenon as mentioned before. Dense\nconnection applies shortcuts among all the layers [6]. In a dense block (a number of convolution layers\nwith dense connection), the input of each layer is the concatenation of all the previous layers\u2019 output.\nFree data \ufb02ow in dense block bene\ufb01ts the robustness of network. Tong et al. [23] applied dense\nconnection by cascading dense blocks for image super-resolution task. Li et al. [12] implemented\ndense connection with U-Net. Although it brings additional network parameters, dense connection is\na worthy trade-off. And in this paper, we will limit the network parameters (like reduce the number\nof intermedia feature channel) to show the superiority.\n\n2.3 Dilated Convolution\n\nYu et al. [31] \ufb01rstly proposed dilated convolution for senmatic segmentation. Receptive \ufb01eld\nhas sensitive connection with network\u2019s abilty of latent global information extraction. In classic\nconvolution, deeper layers involve a combination of receptive \ufb01elds from former layers, while these\nreceptive \ufb01elds have large overlapping area. Dialted convolution applies hollow convolution kernel to\nalleviate overlapping with no more parameters. Moeskops et al. [13] introduced dilated convolution\nto brain MRI segmentation and proved that dilated network has larger receptive \ufb01eld with fewer\nnetwork parameters than fully convolutional network. Perone et al. [15] applied parallel convolution\nwith different dilation scales to abstract multi-scale information. Qiao et al. [35] proposed Pyramid\nDilated Convolution Unit as a birdge to connect the encoder and the decoder of U-Net. Sun et al. [22]\nadopt dilated convolution in cascading blocks.\n\n2Our code is released on GitHub:https://github.com/tinyRattar/CSMRI_0325\n\n2\n\n\f(a)\n\n(b)\n\n(c)\n\n(d)\n\nFigure 1: Sub-Nyquist sampled MR Image. (a) Fully sampled MR Image x. (b) Zero-\ufb01lled recon-\nstructed Image xu from sub-Nyquist sampled k-space data (d). (c) The k-space data of the (a). (d)\nAcquired k-space data y with a 28.5% sampling rate Cartesian mask.\n\n3 Method\n\n3.1 Problem Formulation\n\nThe problem is to reconstruct fully-sampled image from sub-sampled k-space data. With sub-Nyquist\nsampling, the acquisition process can be written as:\n\ny = M (cid:12) F x + \u03b5\n\n(1)\nHere x \u2208 CNx\u00d7Ny is the original MR image (fully-sampled) to be reconstructed and F is Fourier\nTransfrom operator. M \u2208 CNx\u00d7Ny is the sampling mask matrix composed of 1 and 0. The values\nof M stand for the corresponding k-space positions are sampled or not. (cid:12) is pixel-wise multiply\noperation. Notice that sampling style is limited by MRI equipment. In this paper, we focus on\nthe phase direction sampling, i.e. M only contains 0-lines and 1-lines. \u03b5 is the noise generated\nduring acquisition and y \u2208 CNx\u00d7Ny is the k-space data what we actually observed. An example of\nsub-Nyquist sampled MR Image is given in Figure 1.\nUnfortunately, Eq.1 is underdetermined. In order to solve the ill-posed inversion, conventional\nCS-MRI methods formulate an optimisation problem:\n\n\u02c6x = arg min\n\nx\n\n(cid:107)M (cid:12) F x \u2212 y(cid:107)2\n\n2 +\n\n\u03bbi\u03c8i(x)\n\n(2)\n\n(cid:88)\n\ni\n\n\u03c8i is a regularisation term on x, and \u03bbi is a weight to balance the importance of regularisation terms\nand data \ufb01delity. In our deep learning methods, a CNN with leanable parameters is introduced to\nreconstruct x, so the formulation can convert as follows:\n\n\u02c6x = arg min\n\nx\n\n(cid:107)M (cid:12) F x \u2212 y(cid:107)2\n\n2 + \u03bb(cid:107)x \u2212 fcnn(xu|\u03b8)(cid:107)2\n\n2\n\n(3)\n\nhere xu is the zero-\ufb01lled reconstruction calculated by xu = F H y where F H is inverse Fourier\nTranform operator. fcnn represent the forward function of CNN with the parameters \u03b8. In order to\ngenerate images like real fully-sampled ones, the optimization of CNN can be written as:\n\n(cid:107)xj \u2212 fcnn(F H yj|\u03b8)(cid:107)2\n\n2\n\n(4)\n\nN(cid:88)\n\nj\n\n\u02c6\u03b8 = arg min\n\n\u03b8\n\nwith suf\ufb01cient traning data {(xj, yj)|j = 1, 2,\u00b7\u00b7\u00b7 , N} and Stochastic Gradient Descent algorithm,\nCNN can convergence to reasonable state. With \ufb01xed CNN, Eq. 3 can be written as:\n\nx = fdc(xin, y, M )\n\n(5)\nwhere xin = fcnn(F H yj|\u03b8) is the input image reconstructed from CNN. The details of the data\nconsistency layer will be shown in Section 3.5. Furthermore, if the data consistency operation is a\ndetermined function to ensure the data \ufb01delity, we can regard it as a part of CNN. And here comes\nthe formulation of our model:\n\ns.t. M (cid:12) F xdc = y\n\nN(cid:88)\n\n\u02c6\u03b8 = arg min\n\n\u03b8\n\nj\n\n(cid:107)xj \u2212 fdc(fcnn(F H yj|\u03b8), yj, M j)(cid:107)2\n\n2\n\n(6)\n\n3\n\n\f3.2 Proposed Network\n\nWe propose Cascaded Dilated Dense Network with Two-step Data Consistency layer\n(CDDNwithTDC) for MR image reconstruction. Figure 2 shows an overview of our proposed\nnetwork, which is composed of a serial of sub-networks. Each sub-network has a De-Aliase Module\n(DAM) and a Two-step Data Consistency layer (TDC). We use dense block in the DAM and a\ngeometric growth dilation is applied on each dense module for receptive \ufb01eld extension. As MR\ndata is in complex \ufb01eld, we use two channels to represent real part and imaginary part respectively.\nFor example, the input zero-\ufb01lling image xu \u2208 CNx\u00d7Ny is converted to xinput \u2208 R2\u00d7Nx\u00d7Ny. The\ndetails will be described in the following.\n\nFigure 2: Overview of CDDNwithTDC. We use rectangles with colors to indicate different modules,\nwhich is illustrated at the bottom right. A brief illustration of dense block is given at the bottom left.\n\n3.3 De-Aliase Module\n\nDe-Aliase Module (DAM) is used to generate aliase-free images. The input of the \ufb01rst module is\nzero-\ufb01lled MR image while the subsequent modules take the output of former sub-networks as input.\nThe module contains abstraction layer, dense block, transition layer and restore layer. In addition, A\nglobal residual connection is applied.\nThe abstraction layer \ufb01rstly converts the input image xim \u2208 R2\u00d7Nx\u00d7Ny to feature maps xf eature \u2208\nRNf\u00d7Nx\u00d7Ny. The forward operation of dense block can be written as xj = f ([x0, x1,\u00b7\u00b7\u00b7 , xj\u22121])\nwhere f is convolution operation (called dense layer) and xi is the output of ith layer (speci\ufb01cally,\nx0 is the input of dense block). The inputs are concatenated in the dimension of channel. f has two\nparts, the \ufb01rst part is a convolution layer with 1 \u00d7 1 kernel called bottleneck layer, which reduces the\nnumber of feature maps to the original input number (i.e. Nf ). The second part is a convolution layer\nwith 3\u00d7 3 kernel and the outputs have the same number of feature maps. The number (Ng) is called as\ngrowth rate, because the channel of features \"grows\" layer by layer. All the output of dense layers are\nconcatenated and are fed into a convolution layer with 1 \u00d7 1 kernel (i.e. transition layer) for halving\nthe number of feature maps. Finally, the restore layer generate output image xout \u2208 R2\u00d7Nx\u00d7Ny by a\nconvolution layer with 3 \u00d7 3 kernel. Notice that every convolution layer is a combination of recti\ufb01ed\nlinear unit activation (ReLU) [4], batch normalization (BN) [8] and convolution neuron.\nDense connection enables intra-block data \ufb02ow. Such architecture can signi\ufb01cantly bene\ufb01ts the per-\nformance and robustness. We limit the network parameters on purpose to show that the imporvement\nis resulted from network architecture rather than simple parameter increment.\n\n3.4 Dilated Convolution\n\nWith the analogy of the biological term, receptive \ufb01eld descripts the area from where arti\ufb01cial neuron\nabstarct information. In other words, it stands for how large portion of image can be seen by a neuron.\nZero-\ufb01lled MR Images suffer from aliasing artifact. Figure 3 shows an example. With interlaced\nsampling in k-space, the original image occueres in the corrupted image with different offsets in\nimage domain. Notice that additional central phasing-coding lines are fully-sampled as they contains\n\n4\n\n\f(a) ZF\n\n(b) Original\n\nFigure 3: Aliasing artifact phenomenon. The sampling mask is the same as Figure 1(d). We use red\nboxes to mark the same low signal area of the original image which can be found several times in the\naliased image.\n\nnon-sparse low frequency information, so that we can faintly recognize the majority of original image.\nIn order to integrate the scattered many-for-one information, a large receptive \ufb01eld is in need.\nWe implement the dilated convolution with dense block by appling geometrically increasing (i.e.\n1, 2, 4,\u00b7\u00b7\u00b7) dilation scale, Figure 4 gives an illustration of so-called Dilated Dense Block. The\ncombination of dilated convolution and dense connection enables Pyramid-like multi-scale feature\nfusion instead of parallel convolution [15, 35] while keep the depth of network. On the other hand, it\nsuccessfully expands receptive \ufb01eld without any addition in network parameters.\n\nFigure 4: Dilated Dense Block with geometrically increasing dilation scale. The \ufb01nal layer is the\nrestore layer mentioned in Section 3.3, which is 1-dilation convolution to fuse all the former outputs.\nAll the 1 \u00d7 1 convolution are omitted, not only the bottleneck layers but also the transition layer\n\n3.5 Two-step Data Consistency\n\nAs mentioned before, MRI acquires data in k-space. Data consistency in frequency domain is needed.\nWith \ufb01xed parameters \u03b8, Eq.3 has a closed-form solution [18], which can be written as:\n\n(cid:19)\n\n(cid:18)\n\nxdc = FH\n\n(1 \u2212 M ) (cid:12) F xin + M (cid:12) (\n\n1\n\n1 + \u03bb\n\nF xin +\n\n\u03bb\n\n1 + \u03bb\n\ny)\n\n(7)\n\nHere xdc is the result image and 1 is an all-one matrix. It can be seen as a linear combination taken\nbetween y and F x at the valid position of M. Directly replacement is an extreme case with \u03bb = \u221e:\n(8)\n\n(1 \u2212 M ) (cid:12) F xin + M (cid:12) y\n\nxdc = FH\n\n(cid:16)\n\n(cid:17)\n\nUnlike traditional image restoration task, the corrupted data from sub-sampled MRI is exactly true in\nthe sampled location. During reconstruction, we have to ensure the invariance of the true part. Direct\nreplacement can meet the requirement, while it brokes the self-consistency of frequency information.\n\n5\n\n\fIt means the hybird result are unnatural in image-domain. In other word, direct replacement only\ncorrects speci\ufb01c(sampled location) k-space data while leaving others in outdated state.\nIn this paper, we propose a two-step data consistency layer. As shown in Figure 5, we \ufb01rstly replace\ncorresponding phase-coding lines of generated image xin with the original sampled k-space data y.\nThen we convert the result from complex-valued to real-valued format by calculating the modulus\nxm = |xdc|. In the end, another k-space correction is applied on the modulus for data consistency.\nThe two-step data consistency can be formulated as:\n\n(cid:12)(cid:12)(cid:12)FH ((1 \u2212 M ) (cid:12) F xin + M (cid:12) y)\n\n(cid:12)(cid:12)(cid:12) + M (cid:12) y\n\n(cid:17)\n\n(cid:16)\n\nftdc(xin, y, M ) = FH\n\n(1 \u2212 M ) (cid:12) F\n\n(9)\n\nFigure 5: Two-step Data Consistency. The bene\ufb01ts will be evaluated with experiments in Section 4.3.\n\nEmpirical experiments prove the effectiveness as shown in Figure 6(a) and further discussion is taken\nin Section 2 of the Supplementary Material.\n\n4 Experiments\n\n4.1\n\nImplemetation Details\n\nExperiments are implemented using Pytorch platform on four NVIDIA GeForce GTX 1080Ti with\n11GB GPU meomry. Our network is trained with Adam [9] optimizer, initial learning rate is set as\n0.0001, the \ufb01rst momentum is 0.9 and the second momentum is 0.999. Weight decay regularization\nparameter is set as 10\u22127. Batch size is 8 and the network is trained for 1000 epochs to ensure\nconvergence.\nWe cascade 5 sub-networks as default. Each dense block has three BN+ReLU+Conv layers with\n1, 2, 4-dilation, and the growth rate is set as 16. All the convolution layers have 16 feature maps\nexcept the last one for mapping from features to two channel images.\nUnless otherwise stated, other contrastive networks take the same hyper-parameters. Any notable\ndetails will be descripted in the correponding sub-section.\n\n4.2 Dataset\n\nOur dataset, established based on the work of Alexander et al. [1], contains 3300 cardiac real-valued\nMR images from 33 patients. The \ufb01rst 30 patients\u2019 data are training set while the last 3 patients\nare testing set. We use random Cartesian mask with 15% sampling rate like Figure 11(f) as default\nsetting.\n\n4.3\n\nIntra-Method Evaluation\n\nIn this experiment, we compare the proposed CDDNwithTDC with two variants. One is CDNwithDC,\nwhich is implemented without dilated convolution and uses traditional one-step data consistency\nlayer instead. The other network has the Dilated Dense DAM, called CDDNwithDC. We take this\nexperiment to prove the bene\ufb01ts from geometric dilation and two-step data consistency layer. These\nnetworks are trained with 30% random Cartesian mask.\nFigure 6(a) shows the curve of training MSE loss and Figure 6(b) shows the histogram of testing\nresult, which is taken with two measures, peak signal-to-noise ratio (PSNR) and structural similarity\nindex measure (SSIM) [27]. Dilated convolution can abstract latent information from larger receptive\n\ufb01eld without parameters increment and TDC can signi\ufb01cantly imporve accuracy with negligible\ncomputational overhead. Figure 7 gives an example from testing set with 15% sampling rate,\nindicating that network with TDC result in less reconstruction error.\n\n6\n\n\f(a)\n\n(b)\n\nFigure 6: Intra-method comparasion. (a) MSE loss. (b) Testing PSNR/SSIM.\n\n(a) Ground Truth\n\n(b) Zero-\ufb01llied\n\n(c) CDDNwithDC\n\n(d) CDDNwithTDC\n\nFigure 7: Bene\ufb01ts from Two-step Data Consistency layer.\n\n4.4 Inter-Methods Evaluation\n\nWe compare our CDDNwithTDC with deep-learning methods U-Net [7], DC-CNN [21], RDN [22]\nand conventional methods DLMRI [18] and NLR [3]. DC-CNN is re-implemented according to their\npaper. With the way of naming in the original paper, we use D5-C5 for 2D reconstruction. As for\nRDN, we choose the 5B-3D-3R for comparasion, which has the same quantity of network parameters.\nFigure 8 shows the result on 15% random Cartesian mask. We give a dobozdiagram of general result\n(Figure 8(a)) and detailed PSNR on every image of 100 testing set (Figure 8(b)) respectively.\n\n(a)\n\n(b)\n\nFigure 8: The testing result of Inter-Methods Evaluation. As for (b), we show the result from every\nthird image of the 300 testing set, and our method completely exceeds the others.\n\nAs our testing set is composed of only three patients, we take 11-fold cross validation experiments\nin order to alleviate the speci\ufb01city. Our proposed CDDNwithTDC and DCCNN are re-trained\nindividually 10 additional times for further inter-method comparation. In the ith experiment, we take\n(i \u2217 3 \u2212 2),(i \u2217 3 \u2212 1),(i \u2217 3)-th patients\u2019 data as testing set and the remained 30 patients\u2019 data as\ntraining set. Figure. 9 shows the result of cross validation, which proves the robustness.\nDetailed quantitative comparasion for deep methods is given in Table 1. RDN suffers from long\nreconstruction time due to its recursive methods. Our proposed method has fewer parameters\n\n7\n\n\fFigure 9: 11-fold Cross Validation.\n\nthan DC-CNN/RDN and produces better result. We also cascade 10 sub-networks for our method\n(Proposed-C10) to reach a comparable number of parameters. An qualitative comparasion is available\nat Figure 11 as well. We also take experiments on different sampling rate to show the robustness of\nour method, and the quantitative result is given at Table 2.\n\nTable 1: Comparasion of Deep Methods\n\nMethod\nPSNR\n\nNum. of Params.\n\nTrain Time(min/epoch)\n\nTest Time(s/f rame)\n\n1.1\n0.05\n\n35.61\n119k\n5.8\n0.30\n\nU-Net DC-CNN RDN Proposed Proposed-C10\n31.61\n1575k\n\n34.87\n144k\n1.2\n0.05\n\n34.95\n144k\n12.0\n0.65\n\n35.24\n59k\n3.0\n0.17\n\n4.5 Experiment on FastMRI\n\nFastMRI [32] is a dataset of knee MRI. We trained the proposed CDDNwithTDC on part of FastMRI\ndataset (about 6500 single frame as training set and 700 frames as testing set) to demonstrate the\nadaptation in different type of MRI. We use the ESC(emulated single-coil) data as ground truth and\napply randomly generated mask of 25% sampling rate. Figure. 10(a) shows the loss curves and\nFigure 10(b) and 10(c) show a qualitative result. It can be seen that our method can reconstructed\naccuracy details for knee MRI as well as cardiac.\n\n(a) Result on part of FastMRI\n\n(b) GT (ESC)\n\n(c) Rec\n\nFigure 10: Evaluation on FastMRI dataset\n\n5 Conclusion\n\nWe propose a Cascaded Dilated Dense Network with Two-step Data Consistency layer in MRI\nreconstruction. Cascading De-Aliase Module based on dense block results in better performance\nwith fewer parameters. Dilated convolution boost the performance of dense blocks. The proposed\ntwo-step data consistency layer enhances the result in image domain while keep the complete data\nconsistency in k-space. The proposed network achieves state-of-art result and has advantage in the\nnumber of network parameters.\n\n8\n\n\f(a) Ground Truth\n\n(b) PSNR=24.44\n\n(c) PSNR=32.39\n\n(d) PSNR=34.38\n\n(e) PSNR=31.87\n\n(f) Mask\n\n(g) Zero-\ufb01llied\n\n(h) DLMRI\n\n(i) NLR\n\n(j) U-Net\n\n(k) PSNR=36.51\n\n(l) PSNR=36.75\n\n(m) PSNR=36.43\n\n(n) PSNR=36.96\n\n(o) PSNR=37.27\n\n(p) DCCNN\n\n(q) RDN\n\n(r) Proposed-DC\n\n(s) Proposed\n\n(t) Proposed-C10\n\nFigure 11: Qualitative Comparasion. The 1st, 3rd rows are the reconstructed images and 2nd, 4th\nrows are the corresponding residual images.\n\nTable 2: PSNR/SSIM Result with Different Sampling Rate\n\nMethod\nDLMRL\n\nNLR\nU-Net\nDCCNN\n\nRDN\n\nProposed\n\nProposed-C10\n\n2.5%\n\n5%\n\n15%\n\n30%\n\n25.03/0.8179\n27.30/0.8557\n25.96/0.8316\n28.18/0.8872\n28.29/0.8870\n28.43/0.8927\n28.86/0.9029\n\n29.46/0.9017\n31.43/0.9233\n29.45/0.8271\n32.24/0.9430\n31.70/0.9364\n32.55/0.9481\n32.94/0.9526\n\n31.76/0.9350\n32.99/0.9461\n31.58/0.9312\n34.87/0.9649\n34.95/0.9665\n35.30/0.9689\n35.60/0.9713\n\n34.18/0.9548\n36.66/0.9734\n37.24/0.9752\n41.13/0.9900\n40.54/0.9883\n41.66/0.9913\n42.03/0.9920\n\n6 Acknowledgements\n\nThis work is sponsored in part by the Key Project of the National Natural Science Foundation of China under\nGrant 61731009, and in part by the National Natural Science Foundation of China under Grant 61871185, and\nin part by \u201cChenguang Program\" supported by Shanghai Education Development Foundation and Shanghai\nMunicipal Education Commission under Grant 17CG25.\n\n9\n\n\fReferences\n[1] Alexander Andreopoulos and John K. Tsotsos. Ef\ufb01cient and generalizable statistical models of shape and\n\nappearance for analysis of cardiac mri. Medical Image Analysis, 12(3):335\u2013357, 2008.\n\n[2] Emmanuel Candes, Justin Romberg, and Terence Tao. Robust uncertainty principles: Exact signal\n\nreconstruction from highly incomplete frequency information. arXiv preprint math/0409186, 2004.\n\n[3] Weisheng Dong, Guangming Shi, Xin Li, Yi Ma, and Feng Huang. Compressive sensing via nonlocal\n\nlow-rank regularization. IEEE Transactions on Image Processing, 23(8):3618\u20133632, 2014.\n\n[4] Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse recti\ufb01er neural networks. In Proceedings\n\nof the fourteenth international conference on arti\ufb01cial intelligence and statistics, pages 315\u2013323, 2011.\n\n[5] Elma Hot and Petar Sekuli\u00b4c. Compressed sensing mri using masked dct and dft measurements. In 2015\n\n4th Mediterranean Conference on Embedded Computing (MECO), pages 323\u2013326. IEEE, 2015.\n\n[6] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected\nconvolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition,\npages 4700\u20134708, 2017.\n\n[7] Chang Min Hyun, Hwa Pyung Kim, Sung Min Lee, Sungchul Lee, and Jin Keun Seo. Deep learning for\n\nundersampled MRI reconstruction. Physics in Medicine & Biology, 63(13):135007, 2018.\n\n[8] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing\ninternal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning, ICML\n2015, Lille, France, 6-11 July 2015, pages 448\u2013456, 2015.\n\n[9] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint\n\narXiv:1412.6980, 2014.\n\n[10] Zongying Lai, Xiaobo Qu, Yunsong Liu, Di Guo, Jing Ye, Zhifang Zhan, and Zhong Chen.\n\nImage\nreconstruction of compressed sensing MRI using graph-based redundant wavelet transform. Medical image\nanalysis, 27:93\u2013104, 2016.\n\n[11] PC Lauterber. Image formation by induced local interactions: examples employing nuclear magnetic\n\nresonance. Nature, 246:469, 1974.\n\n[12] Xiaomeng Li, Hao Chen, Xiaojuan Qi, Qi Dou, Chi-Wing Fu, and Pheng-Ann Heng. H-denseunet: hybrid\ndensely connected unet for liver and tumor segmentation from ct volumes. IEEE transactions on medical\nimaging, 37(12):2663\u20132674, 2018.\n\n[13] Pim Moeskops, Mitko Veta, Maxime W Lafarge, Koen AJ Eppenhof, and Josien PW Pluim. Adversarial\ntraining and dilated convolutions for brain mri segmentation. In Deep learning in medical image analysis\nand multimodal learning for clinical decision support, pages 56\u201364. Springer, 2017.\n\n[14] Harry Nyquist. Certain topics in telegraph transmission theory. Proceedings of the IEEE, 90(2):280\u2013305,\n\n1928.\n\n[15] Christian S Perone, Evan Calabrese, and Julien Cohen-Adad. Spinal cord gray matter segmentation using\n\ndeep dilated convolutions. Scienti\ufb01c reports, 8(1):5966, 2018.\n\n[16] Xiaobo Qu, Di Guo, Bende Ning, Yingkun Hou, Yulan Lin, Shuhui Cai, and Zhong Chen. Undersampled\nmri reconstruction with patch-based directional wavelets. Magnetic resonance imaging, 30(7):964\u2013977,\n2012.\n\n[17] Tran Minh Quan, Thanh Nguyen-Duc, and Won Ki Jeong. Compressed sensing MRI reconstruction with\n\ncyclic loss in generative adversarial networks. IEEE Transactions on Medical Imaging, PP(99), 2017.\n\n[18] Saiprasad Ravishankar and Yoram Bresler. Mr image reconstruction from highly undersampled k-space\n\ndata by dictionary learning. IEEE transactions on medical imaging, 30(5):1028\u20131041, 2010.\n\n[19] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical\nimage segmentation. In International Conference on Medical image computing and computer-assisted\nintervention, pages 234\u2013241. Springer, 2015.\n\n[20] Lingala Sajan Goud and Jacob Mathews. Blind compressive sensing dynamic mri. IEEE Trans Med\n\nImaging, 32(6):1132\u20131145, 2013.\n\n10\n\n\f[21] Jo Schlemper, Jose Caballero, Joseph V Hajnal, Anthony Price, and Daniel Rueckert. A deep cascade of\nconvolutional neural networks for mr image reconstruction. In International Conference on Information\nProcessing in Medical Imaging, pages 647\u2013658. Springer, 2017.\n\n[22] Liyan Sun, Zhiwen Fan, Yue Huang, Xinghao Ding, and John Paisley. Compressed sensing mri using a\n\nrecursive dilated network. In Thirty-Second AAAI Conference on Arti\ufb01cial Intelligence, 2018.\n\n[23] Tong Tong, Gen Li, Xiejie Liu, and Qinquan Gao. Image super-resolution using dense skip connections.\n\nIn Proceedings of the IEEE International Conference on Computer Vision, pages 4799\u20134807, 2017.\n\n[24] Shanshan Wang, Zhenghang Su, Leslie Ying, Xi Peng, Shun Zhu, Feng Liang, Dagan Feng, and Dong\nLiang. Accelerating magnetic resonance imaging via deep learning. In 2016 IEEE 13th International\nSymposium on Biomedical Imaging (ISBI), pages 514\u2013517. IEEE, 2016.\n\n[25] Yanhua Wang and Leslie Ying. Undersampled dynamic magnetic resonance imaging using kernel principal\ncomponent analysis. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine\nand Biology Society, pages 1533\u20131536. IEEE, 2014.\n\n[26] Zhaowen Wang, Ding Liu, Jianchao Yang, Wei Han, and Thomas Huang. Deep networks for image\nsuper-resolution with sparse prior. In Proceedings of the IEEE international conference on computer vision,\npages 370\u2013378, 2015.\n\n[27] Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al. Image quality assessment: from\n\nerror visibility to structural similarity. IEEE transactions on image processing, 13(4):600\u2013612, 2004.\n\n[28] Ruomei Yan and Ling Shao. Blind image blur estimation via deep learning. IEEE Transactions on Image\n\nProcessing, 25(4):1910\u20131921, 2016.\n\n[29] Guang Yang, Simiao Yu, Hao Dong, Greg Slabaugh, Pier Luigi Dragotti, Xujiong Ye, Fangde Liu, Simon\nArridge, Jennifer Keegan, Yike Guo, et al. Dagan: deep de-aliasing generative adversarial networks for fast\ncompressed sensing MRI reconstruction. IEEE transactions on medical imaging, 37(6):1310\u20131321, 2017.\n\n[30] Yan Yang, Jian Sun, Huibin Li, and Zongben Xu. Admm-csnet: A deep learning approach for image\n\ncompressive sensing. IEEE transactions on pattern analysis and machine intelligence, 2018.\n\n[31] Fisher Yu and Vladlen Koltun. Multi-scale context aggregation by dilated convolutions. arXiv preprint\n\narXiv:1511.07122, 2015.\n\n[32] Jure Zbontar, Florian Knoll, Anuroop Sriram, Matthew J. Muckley, Mary Bruno, Aaron Defazio, Marc\nParente, Krzysztof J. Geras, Joe Katsnelson, Hersh Chandarana, Zizhao Zhang, Michal Drozdzal, Adriana\nRomero, Michael Rabbat, Pascal Vincent, James Pinkerton, Duo Wang, Na\ufb01ssa Yakubova, Erich Owens,\nC. Lawrence Zitnick, Michael P. Recht, Daniel K. Sodickson, and Yvonne W. Lui. fastMRI: An open\ndataset and benchmarks for accelerated MRI. 2018.\n\n[33] Zhifang Zhan, Jian-Feng Cai, Di Guo, Yunsong Liu, Zhong Chen, and Xiaobo Qu. Fast multiclass\ndictionaries learning with geometrical directions in mri reconstruction. IEEE Transactions on Biomedical\nEngineering, 63(9):1850\u20131861, 2015.\n\n[34] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a gaussian denoiser:\nResidual learning of deep cnn for image denoising. IEEE Transactions on Image Processing, 26(7):3142\u2013\n3155, 2017.\n\n[35] Qiao Zhang, Zhipeng Cui, Xiaoguang Niu, Shijie Geng, and Yu Qiao. Image segmentation with pyramid\nIn International Conference on Neural Information\n\ndilated convolution based on resnet and u-net.\nProcessing, pages 364\u2013372. Springer, 2017.\n\n11\n\n\f", "award": [], "sourceid": 1010, "authors": [{"given_name": "Hao", "family_name": "Zheng", "institution": "East China Normal University"}, {"given_name": "Faming", "family_name": "Fang", "institution": "East China Normal University"}, {"given_name": "Guixu", "family_name": "Zhang", "institution": "East China Normal University"}]}