This statement occurs four times in the King James Version. In each context more meaning is added to it. Basically, it means that we not only begin our walk with God by exercising faith toward God, but we continue walking with Him in the same way. Faith, like repentance, is something we will go on using the rest of our lives. Once the foundation has been laid, faith becomes our new principle of life. The basic attitude of faith is trustful receptivity. We continue to receive from God in this way.  

As ye have therefore received Christ Jesus the Lord, so walk ye in him: Rooted and built up in him, and established in the faith, as ye have been taught, abounding therein with thanksgiving (Colossians 2:6, 7).  

 Living by faith means living expectantly toward God. 

We have not received all of God’s promises yet, but what we have received should strengthen our hope and anticipation for what is to come. What God has promised all of us as His children, and what He has spoken to us individually, will surely come to pass if we wait for it. Waiting means more than allowing time to pass; it means actively adhering to the promises, giving ourselves to them, and allowing them to shape our expectation. 

For the vision is yet for an appointed time, but at the end it shall speak, and not lie: though it tarry, wait for it; because it will surely come, it will not tarry. Behold, his soul which is lifted up [swollen with impatient pride] is not upright in him: but the just shall live by his faith (Habakkuk 2:3, 4). 

Living by faith means growing in reliance on God as knowledge of Him increases. 

The more revelation of God we receive through His Word, the more we have to lean upon or trust. Each time we experience the faithfulness of God to His promises, the easier it becomes to anticipate that He will fulfill the other things He has said He will do for us. Faith begets faith. The gospel plays no favorites.  

For I am not ashamed of the gospel of Christ: for it is the power of God unto salvation to every one that believeth; to the Jew first, and also to the Greek. For therein is the righteousness of God revealed from faith to faith: as it is written. The just shall live by faith (Romans 1:16, 17).  

Living by faith means release from all self-effort. 

We no longer attempt to add anything to the merits of the blood of Jesus Christ. What God accepts is good enough for us. We allow the blood to speak peace within and are not tricked into some subtle means of trying to earn God’s grace.  

But that no man is justified by the law in the sight of God, it is evident: for. The just shall live by faith. And the law is not of faith: but, The man that doeth them shall live in them (Galatians 3:11, 12). 

Living by faith is mixing believing with patience. 

One of the hardest pressures to endure is that of waiting a long time. In fact, such waiting does not come naturally to man. It is a fruit of the Spirit called long-suffering or patient endurance. Allowing God to develop this characteristic in us causes us to have a strong anchor when it comes to times of testing. We are able to hold steady in God, and at the same time maintain boldness in prayer. 

Cast not away therefore your confidence, which hath great recompense of reward. For ye have need of patience, that, after ye have done the will of God, ye might receive the promise. For yet a little while, and he that shall come will come, and will not tarry. Now the just shall live by faith: but if any man draw back, my soul shall have no pleasure in him (Hebrews 10:35-38).